System and method for image registration

ABSTRACT

A system and method for providing processable data associated with anatomical images may provide a user interface including a pivot and stem tool via which to align multiple images, a flashlight bar for viewing portions of an overlaid image, and/or user-movable markers for establishing a location of one or more anatomical landmarks with regions of one or more images.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application claims the benefit, under 35 U.S.C. §119(e), of U.S.Provisional Patent Application Ser. Nos. 61/468,884, 61/468,887,61/468,891, 61/468,897 and 61/468,901, filed Mar. 29, 2011, the entirecontents of which are hereby incorporated by reference.

FIELD OF THE INVENTION

The present invention relates to a system and method for surgicalplanning for implanting a stimulation leadwire, registration of imagesfor generating a model via which to determine how to apply astimulation, for programming stimulation settings, and/or for applying atherapeutic stimulation, and/or for integration of components providingsuch functionality.

BACKGROUND

Electrical stimulation of an anatomical region, e.g., deep brainstimulation (DBS), such as of the thalamus or basal ganglia, is aclinical technique for the treatment of disorders such as essentialtremor, Parkinson's disease (PD), and other physiological disorders. DBSmay also be useful for traumatic brain injury and stroke. Pilot studieshave also begun to examine the utility of DBS for treating dystonia,epilepsy, and obsessive-compulsive disorder.

A stimulation procedure, such as DBS, typically involves first obtainingpreoperative images, e.g., of the patient's brain, such as by using acomputed tomography (CT) scanner device, a magnetic resonance imaging(MRI) device, or any other imaging modality. This sometimes involvesfirst affixing to the patient's skull spherical or other fiducialmarkers that are visible on the images produced by the imaging modality.The fiducial markers help register the preoperative images to the actualphysical position of the patient in the operating room during the latersurgical procedure.

After the preoperative images are acquired by the imaging modality, theyare then loaded onto an image-guided surgical (IGS) workstation, and,using the preoperative images displayed on the IGS workstation, aneurosurgeon can select a target region, e.g., within the brain, anentry point, e.g., on the patient's skull, and a desired trajectorybetween the entry point and the target region. The entry point andtrajectory are typically carefully selected to avoid intersecting orotherwise damaging certain nearby critical structures or vasculature,e.g., of the brain.

In the operating room, the physician marks the entry point on thepatient's skull, drills a burr hole at that location, and affixes atrajectory guide device about the burr hole. The trajectory guide deviceincludes a bore that can be aimed to obtain the desired trajectory tothe target region. After aiming, the trajectory guide is locked topreserve the aimed trajectory toward the target region. After the aimedtrajectory has been locked in using the trajectory guide, a microdriveintroducer is used to insert the surgical instrument along thetrajectory toward the target region, e.g., of the brain. The surgicalinstrument may include, among other things, a recording electrodeleadwire, for recording intrinsic electrical signals, e.g., of thebrain; a stimulation electrode leadwire, for providing electrical energyto the target region, e.g., of the brain; or associated auxiliaryguidewires or guide catheters for steering a primary instrument towardthe target region, e.g., of the brain.

The stimulation electrode leadwire, which typically includes multipleclosely-spaced electrically independent stimulation electrode contacts,is then introduced to deliver the therapeutic stimulation to the targetregion, e.g., of the brain. The stimulation electrode leadwire is thenimmobilized, such as by using an instrument immobilization devicelocated at the burr hole entry, e.g., in the patient's skull, in orderfor the DBS therapy to be subsequently performed.

The subthalamic nucleus (STN) represents the most common target for DBStechnology. Clinically effective STN DBS for PD has typically usedelectrode contacts in the anterior-dorsal STN. However, STN DBS exhibitsa low threshold for certain undesirable side effects, such as tetanicmuscle contraction, speech disturbance and ocular deviation. Highlyanisotropic fiber tracks are located about the STN. Such nerve tracksexhibit high electrical conductivity in a particular direction.Activation of these tracks has been implicated in many of the DBS sideeffects. However, there exists a limited understanding of the neuralresponse to DBS. The three-dimensional (3-D) tissue medium near the DBSelectrode typically includes both inhomogeneous and anisotropiccharacteristics. Such complexity makes it difficult to predict theparticular volume of tissue influenced by DBS.

After the immobilization of the stimulation electrode leadwire, theactual stimulation therapy is often not initiated until after a timeperiod of about two-weeks to one month has elapsed. This is dueprimarily to the acute reaction of the brain tissue to the introducedelectrode leadwire (e.g., the formation of adjacent scar tissue), andstabilization of the patient's disease symptoms. At that time, aparticular one or more of the stimulation electrode contacts is selectedfor delivering the therapeutic stimulation, and other stimulationparameters are adjusted to achieve an acceptable level of therapeuticbenefit.

A system and method may estimate stimulation volumes, and display modelsof a patient anatomy and/or a stimulation leadwire, via which tographically identify the estimated stimulation volumes and how theyinteract with various regions of the patient anatomy, for example, asdescribed in U.S. patent application Ser. No. 12/454,330, filed May 15,2009 (“the '330 application”), U.S. patent application Ser. No.12/454,312, filed May 15, 2009 (“the '312 application”), U.S. patentapplication Ser. No. 12/454,340, filed May 15, 2009 (“the '340application”), U.S. patent application Ser. No. 12/454,343, filed May15, 2009 (“the '343 application”), and U.S. patent application Ser. No.12/454,314, filed May 15, 2009 (“the '314 application”), the content ofeach of which is hereby incorporated herein by reference in itsentirety.

SUMMARY

Example embodiments of the present invention provide a system thatincludes modules providing respective user interfaces via which toperform surgical planning, image and atlas registration, and stimulationprogramming. The user interfaces may be graphical user interfaces (GUI)displayed in a display device. The display device may be any suitablyappropriate display device.

Embodiments of the present invention facilitate image registration usedfor accurate modeling of the patient anatomy, stimulation leadwire,estimated stimulation volumes, and interactions of stimulation volumeswith the patient anatomy.

Various systems, system components, and/or program modules may be usedfor performance of various tasks associated with, or that provide anoutput usable for, providing therapeutic stimulation. Embodiments of thepresent invention provide for communication and/or between the varioussystems, system components, and/or program modules.

Example embodiments of the present invention provide methods by which toselect target areas to stimulate, target stimulation parameters, and/ortarget stimulation hardware.

An example embodiment of the present invention provides a method bywhich to output estimated volumes of activation (VOAs) in a shortprocessing time.

The various methods described herein may be practiced, each alone, or invarious combinations.

An example embodiment of the present invention is directed to aprocessor, which may be implemented using any conventional processingcircuit and device or combination thereof, e.g., a Central ProcessingUnit (CPU) of a Personal Computer (PC) or other workstation processor,to execute code provided, e.g., on a hardware computer-readable mediumincluding any conventional memory device, to perform any of the methodsdescribed herein, alone or in combination. The memory device may includeany conventional permanent and/or temporary memory circuits orcombination thereof, a non-exhaustive list of which includes RandomAccess Memory (RAM), Read Only Memory (ROM), Compact Disks (CD), DigitalVersatile Disk (DVD), and magnetic tape.

An example embodiment of the present invention is directed to a hardwarecomputer-readable medium, e.g., as described above, having storedthereon instructions executable by a processor to perform the methodsdescribed herein.

An example embodiment of the present invention is directed to a method,e.g., of a hardware component or machine, of transmitting instructionsexecutable by a processor to perform the methods described herein.

BRIEF DESCRIPTION OF THE DRAWINGS

In the drawings, which are not necessarily drawn to scale, like numeralsmay describe similar components in different views. The drawingsillustrate generally, by way of example, but not by way of limitation,various embodiments discussed in the present document.

FIG. 1 is a screen shot showing a pivot and stem tool, according to anexample embodiment of the present invention.

FIG. 2 is a screen shot showing markers for illustrating cross-sectionrelationships between orthogonal image slices, according to an exampleembodiment of the present invention.

FIG. 3 is a screen shot showing a flashlight tool for showing regions ofan image, according to an example embodiment of the present invention.

FIG. 4 is a screen shot of a user interface via which to mark ananterior commissure

(AC) and posterior commissure (PC), according to an example embodimentof the present invention.

FIG. 5 is a screen shot showing a magnification tool, according to anexample embodiment of the present invention.

FIG. 6 is a flowchart that illustrates steps for auto-correction for anMR image, according to an example embodiment of the present invention.

FIG. 7 shows how a histogram may be adjusted in an auto-correctionmethod for correcting an MR image, according to an example embodiment ofthe present invention.

FIG. 8 is a flowchart that illustrates steps for auto-correction of a CTimage, according to an example embodiment of the present invention.

FIG. 9 is a screen shot showing a user interface via which to identify amid-sagittal plane (MSP), according to an example embodiment of thepresent invention.

FIG. 10 shows how a line of points may fall within differently angledplanes.

FIG. 11 is a flowchart that illustrates steps for scaling and aligningan atlas to conform to image data, according to an example embodiment ofthe present invention.

FIG. 12 shows relative scaling amounts for an anisotropic atlas scaling,according to an example embodiment of the present invention.

FIG. 13 is a flowchart that illustrates steps for registering a patientpopulation image atlas to a current patient image, according to anexample embodiment of the present invention.

FIG. 14 is a flowchart that illustrates a method for obtaining apatient-specific atlas, according to an example embodiment of thepresent invention.

FIG. 15 shows a series of images representing axial CT slices thatinclude features corresponding to cross sections of leadwires, accordingto an example embodiment of the present invention.

FIG. 16 shows a series of images representing sagittal CT slices thatinclude features corresponding to cross sections of leadwires, accordingto an example embodiment of the present invention.

FIG. 17 shows a headframe including an arc and ring that define atrajectory of a leadwire, according to an example embodiment of thepresent invention.

FIG. 18 shows a representative CT slice that includes features formed bya plurality of directional electrodes in a leadwire, according to anexample embodiment of the present invention.

FIG. 19 shows a representative CT slice that includes features formed bya non-electrode in the leadwire of FIG. 18, according to an exampleembodiment of the present invention.

FIG. 20 shows a composite image formed by combining the representativeslices of FIGS. 18 and 19, according to an example embodiment of thepresent invention.

FIG. 21 shows a profile view of a leadwire, according to an exampleembodiment of the present invention.

FIG. 22 shows a block diagram of a system, according to an exampleembodiment of the present invention.

FIG. 23 shows a single leadwire positioned relative to a target volumeof activation, according to an example embodiment of the presentinvention.

FIG. 24 shows a pair of leadwires positioned relative to a target volumeof activation, according to an example embodiment of the presentinvention.

FIG. 25 shows an image of a maximum volume of activation displayedaccording to an example embodiment of the present invention.

FIG. 26 shows a block diagram of a patient registration system,according to an example embodiment of the present invention.

FIG. 27 is a screen shot showing a highlighting of an anatomicallysignificant portion of a flashlight region, according to an exampleembodiment of the present invention.

DETAILED DESCRIPTION Fuse Images Using a Pivot and Stem TransformationTool and See Live Updates from all Three Planes

It may be advantageous or necessary to correctly position relative toeach other in a system memory various relevant features of a patientanatomy, or features positioned relative to the a patient anatomy. Suchrelative positioning may be useful, for example, for correctlyoutputting a graphical depiction of such features, on which basis aclinician may determine how to program stimulation settings, and/or forthe system to accurately determine stimulation settings to use,estimated VOAs, and/or target VOAs. Different subsets of such featuresmay be identifiable in different patient images, which may be, forexample, of different imaging modalities. For example, certain featuresmay be readily identifiable in a magnetic resonance (MR) image, whileother features may be more readily identifiable in a CT image. Forexample, certain anatomical structures may be more readily identifiablein an MR image than in a CT image, while the reverse may be true of animplanted leadwire. It may therefore be required to correctly registerto each other two or more images, e.g., of different imaging modalities,in order to correctly position the various features relative to eachother.

Medical images, e.g., of two (or more) different modalities, e.g., MR,CT, DTI, PET, Fluoroscopy, two different MR types (T1 MR and T2 MR), ortwo different images of the same modality taken at different times etc.,may be displayed overlaying each other. In an example embodiment, auser-interactive sliding scale may be provided, where a control may beshifted between first and second edges of a slide bar, where the firstedge corresponds to one of the images, and the second edge correspondsto the other image. The closer the user shifts the control towards thefirst edge, the more of first image is represented in the merged displayand the less of the second image is represented in the merged display.The closer the user shifts the control towards the second edge, the moreof second image is represented in the merged display and the less of thefirst image is represented in the merged display. For example, at a leftedge, only the MR would be shown, at the right edge, only the CT wouldbe shown, and at a center point, both images would be equallyrepresented.

In an example embodiment of the present invention, the system mayprovide a selectable option for presenting the two images in acheckerboard pattern including a plurality of image blocks, where foreach pair of adjacent ones of the plurality of image blocks, a portionof the first image is revealed in one of the blocks of the pair and aportion of the second image is revealed in the other of the blocks ofthe pair, as though the portion of the first image displayed in the oneblock obstructs the view of the portion of the second image that isadjacent the portion of the second image that is displayed in the otherof the blocks of the pair, that would have otherwise been displayed.

In an example embodiment, a user interface pivot and stem tool may beprovided via interaction with which the user may change the alignment ofthe images relative to each other.

A screen may be divided into 3 panes to show the overlaid images in,respectively, the axial, coronal, and sagittal views, as shown inFIG. 1. In alternative example embodiments, a screen may be divided intofurther panes to show additional views. FIG. 1 shows an example screenshot 100 of a user interface including an axial pane 102, a coronal pane104, and a sagittal pane 106. In an example embodiment of the presentinvention, when the user interacts with a tool for modifying theregistration of the overlaid images to each other in one of the panes,the system displays a cross-section indicator, e.g., in the form of aline, in each of the other two panes at a location of which the image,in the active pane in which the user is modifying the registration, is across-section. For example, the user may be modifying registration ofaxial images, which correspond to a cross-section view of a particularpoint in the coronal and sagittal images of the other two panes. Suchinformation conveyed by the cross-section indicator is useful to theuser to identify what is being viewed in the active pane by providingthe user visual information regarding the surroundings of the slice towhich the user is navigating in the active pane. FIG. 2 shows an examplescreen shot 200, where the coronal pane 104 is active and cross-sectionindicators 202 and 204 are displayed, respectively, in axial pane 102and sagittal pane 106.

In an alternative example embodiment, the line in the non-active panesshowing the correspondence to the display in the active pane may bedisplayed in response to interaction with a user interface control forchanging the viewed image slice in the active pane, and is removed inresponse to release of the control. For example, FIG. 2 shows an axialimage slice navigator slider control 206 for navigating betweendifferent axial slices, a coronal image slice navigator slider control208 for navigating between different coronal slices, and a sagittalimage slice navigator slider control 210 for navigating betweendifferent sagittal slices. The user may select one of the slidercontrols 206, 208, 210, and slide the respective control to change theviewed image slice in the respective view. Alternatively, the user mayselect a control button located at a far edge of a bar along which theslider control is slidable (e.g., a left pointing arrow and a rightpointing arrow), in response to which selection the system maycorrespondingly continuously slide the slider control toward theselected control button until the user releases the control button.

For each displayed image slice, the system may, in each pane, identify,e.g., immediately above a slider bar in which the slider control isslidable as shown in FIG. 2, a distance the anatomical portion to whichthe displayed image corresponds is from a predetermined origincoordinate. For example, a point that approximates the center of thebrain may be used by the system as the origin. When the user manipulatesone of the slider controls to change the viewed slice in a respectiveactive pane, the system may update a distance identification 215 of theactive pane to reflect the distance of the anatomical regioncorresponding to the newly displayed image slice from the origin. Forexample, should the user slide the slider control 208 of the sagittalpane to the right, the system may modify the distance identification 215in the sagittal pane to indicate a positive number of millimeters, and,should the user slide the slider control 208 of the sagittal pane to theleft, the system may modify the distance identification 215 in thesagittal pane to indicate a negative number of millimeters. Accordingly,upon selection of any of the slider controls 206, 208, 210 in arespective one of the panes, the system may display the cross-sectionindicators in the remaining panes, and upon release of the selectedslider control, may remove the cross-section indicators from thoseremaining panes.

The user may perform the registration modifications in any of the 3panes 102, 104, 106. Indeed, the user may switch between all of the 3panes to perform parts of the registration modification.

One of the tools the user may use to modify the image registration is apivot and stem tool 110, which may be displayed in each of the panes102, 104, 106. For example, the system may display the pivot and stemtool 110 in response to selection of a graphical input object, such as abutton in a toolbar, shown in FIG. 1 as “xform” graphical button 115,which may be displayed in a tool bar. The pivot and stem tool 110 mayinclude a stem associated by the system with one of the two images to beregistered. For example, FIG. 1 shows an overlaid CT image and MR image.The stem may be associated with the CT image or the MR image. In anexample embodiment, the system may be configured to receive user-inputselecting with which of the two images the stem is to be associated. Thestem may extend from a center of the associated image and/or a center ofa significant object within the image. For example, the user may placethe first stem edge of the stem at a point within the image which theuser perceives as being the center of the object. Alternatively, thesystem may automatically detect the significant object and place thefirst stem edge at the center of the detected significant object. Forexample, the image may be a CT of a patient's brain, in which an area ofthe image corresponding to matter as dense as the skull or denser issaturated, i.e., the upper limit of used pixel values is used for allsuch matter. The system may detect where such values lie in the imageand match a formed structure to that which corresponds most closely to atemplate of a skull, thereby setting such structure as the significantobject at the center of which to set the first edge of the stem.Alternatively, the system may automatically place the stem edge at thecenter of the image. The stem may extend from the first edge at thecenter and outwards to a second stem edge.

In an example embodiment, the user may select the first edge associatedwith the center and drag it, in response to which the systemtranslationally shifts the associated image relative to the underlyingimage. For example, the user may move a pointer to the first edge usingany suitably appropriate input device, such as a computer mouse, andselect it, e.g., via a click or via a different input, such as via akeyboard, and drag it, e.g., by correspondingly dragging the computermouse or another input device. A stylus or even a finger may instead beused, e.g., where touch-screen functionality is provided. Any suitablyappropriate input device or combinations thereof usable for apoint-click-drag operation may be used.

In an alternative example embodiment, in response to shifting the firstedge, the center of rotation (as described below) may be shiftedrelative to the image that is rotated via interaction with the pivot andstem tool. According to this embodiment, the system may provide forshifting one of the images relative to the other by user selection ofany point of the displayed images (e.g., other than the interactiveelements of the pivot and stem tool) and a subsequent drag. For example,the system may associate the selection with one of the images, and, inresponse to the drag while the image is selected, the system may shiftthe associated image relative to the other image.

The user may also select the second edge and drag it to the right or theleft of the stem, in response to which the system may rotate theassociated image relative to the underlying image. For example, if theuser drags the second stem edge to the right, the system may rotate theassociated image clockwise, while, if the user drags the second stemedge to the left, the system may rotate the associated imagecounter-clockwise. For example, FIG. 2 shows the stem control 110 inaxial pane 102 after a clockwise shift of the stem control 110 or shiftto the right of the stem control 110 in the axial pane 102, andcorrespondingly shows the clockwise rotation of the MR image relative tothe CT image in the axial pane 102. It is noted that in response to arotational and/or translational shift of one of the images relative tothe other in one of the panes, the mages in the other panes may becorrespondingly changed.

The user may also select the second edge and drag it in-line with thestem inwards towards the first edge or outwards further away from thefirst edge, in response to which the system may correspondingly shortenor lengthen the stem. The length of the stem may impact the rotationalshift of the associated image in response to subsequent dragging of thesecond edge to the right or left. For example, the longer the stem, theless the rotational shift of the image in response to the left or rightdrag because, the further from the center of the rotation, the greaterthe distance that must be covered for a particular angular change. Theuser may therefore desire to work with a longer stem for a preciserotational shift, or a shorter stem for a quick rotational shift.

It is noted that the image with which the stem is associated may be theunderlying image and the stationary image the overlying image, or viceversa.

Overlay Controls for an Overlay View Mode During Registration

As noted above, DTI, PET, Fluoroscopy, two different MR types (T1 MR andT2 MR), etc., may be co-registered. In an example embodiment of thepresent invention, the system may display a flashlight bar 315, as shownin screen shot 300 of FIG. 3, above the images in each of the panes 102,104, 106. With respect to each respective one of the panes 102, 104,106, a first one of the images may be displayed in the viewing paneexcept at a section extending downwards from the bar 315, e.g., along anentire or approximately an entire length of the bar 315, at whichsection the second of the images is displayed. The bar 315 may thereforefunction as a flashlight for shining onto the second image, allowing itto come into view under the bar 315. While the bar 315 is described asbeing at the top of the pane and shining downwards, the bar 315 maysimilarly be positioned instead at the bottom of the pane and shineupwards or may be positioned at one of the right and left sides of thepane and shine towards the opposite side.

The user may select the bar 315 and drag it, for example, using anysuitably appropriate input device that provides a point-click-dragfunctionality. In response to such user input, the system maycorrespondingly shift the bar 315. For example, where the bar 315 ispositioned at the top of the pane, the system may shift the bar 315 tothe right or left. The images may remain stationary with respect totheir positions relative to each other and an area corresponding to theentirety of the images may remain fixed relative to the display panewhile the bar 315 shifts. The region of the second image on which thebar 315 shines for bringing the respective region into view, however,may correspondingly shift in response to the shifting of the bar. Forexample, while the bar 315 is at a center position of the pane, a regionof the second image that is at the center of the pane may come intoview, and when the bar 315 is shifted to the right, the center region(or a portion of the center region) may move out of view while a regionto the right, not previously in view, correspondingly comes into view.The bars 315 in each pane may be independently operated.

For example, in FIG. 3, the image portions displayed in the regionscorresponding to the bars 315 of panes 102, 104, 106 are of CT images,while remaining image regions of the panes show portions of MR images.

The bars may also include a respective bar sizing control 320 at eachedge of the bar, which the user may select and drag, e.g., using anysuitably appropriate input device providing point-click-dragfunctionality (or any other suitable input device), to lengthen orshorten the respective bar 315, and the corresponding region of thesecond image which is shown. For example, if the bar 315 is placed atthe top of the pane, the bar sizing control 320 at the right edge may beselected and dragged either right to lengthen the bar 315 or left toshorten the bar 315; and the bar sizing control 320 at the left edge maybe selected and dragged either to the right to shorten the bar 315 or tothe left to lengthen the bar 315.

In an example embodiment of the present invention, the system mayhighlight regions within the section of the display corresponding to theflashlight bar 315 predefined as significant anatomical regions. Forexample, FIG. 27 shows a highlight region 2700, in which a skull line ina CT image is highlighted, which may help the user determine how wellthe two images are aligned.

In an example embodiment of the present invention, the system maydisplay the bar 315 and associated bas sizing controls 320, and theassociated views of the images, in response to selection of a graphicalinput object, such as a button in a toolbar, shown in FIG. 3 as“overlay” graphical button 330, which may be displayed in a tool bar.

In an example embodiment, the bar 315 and associated controls 320 andimage views may be provided in combination with the pivot and stemcontrol 110. For example, if the user selects the “xform” button 115while in the overlay mode entered into in response to the selection ofthe “overlay” button 330, the pivot and stem control 110 may bedisplayed and operated in the overlay mode.

Drag and Drop Marker Placement

In an example embodiment of the present invention, the system may beconfigured to record structural information regarding anatomicalstructures represented in the medical images. The system may then usethe recorded anatomical structure information concerning the images forregistering other anatomically related objects to the medical images.For example, an anatomical atlas or other volumes, such as those of aDTI atlas or other medical images obtained from a patient population maybe registered to the patient's medical images in accordance with therecorded anatomical structure information.

In an example embodiment of the present invention, the system mayprovide a user interface via which a user may provide input identifyingcertain anatomical landmarks usable by the system for performing suchlater registration to other anatomically related objects. For example,the other anatomically related objects may be warped to represent ananatomical structure whose corresponding landmarks are positioned in amanner that corresponds to the positioning of the landmarks identifiedby the user, as described in further detail below.

In an example embodiment of the present invention, a marker may beassociated by the system with such predefined landmarks. For example, amarker may be associated by the system with the anterior commissure (AC)and another marker may be associated by the system with the posteriorcommissure (PC). While the discussion below refers to the AC and PC, thedescribed features may be applied to other predefined, e.g.,anatomically significant, landmarks. The markers may be displayeddifferently in the displays so that the user can identify one of themarkers as being associated with the AC and the other as beingassociated with the PC. For example, the system may label one of themarkers “AC” and the other marker “PC” and/or the markers may bedisplayed using different shapes or colors (or may be visuallydistinguished in another suitably appropriate manner). For example theAC marker may be orange and the PC marker may be blue. The markers maybe selectable and draggable. The user may select each of the markers andplace it in a displayed MR image, for example, in any one of the threepanes corresponding to the axial, coronal, and sagittal views. Inresponse, the system may also display AC and PC markers in the other twopanes at positions corresponding to the user placement of the markers inthe first pane.

FIG. 4 shows an example screen shot 400 in which an AC marker 402 and aPC marker 404 are displayed in those of panes 102, 104, 106 displayingimage slices of anatomical regions to which the user anchored therespective markers 402 and 404. For example, the markers 402 and 404 mayrepresent a three-dimensional volume. In response to positioning, by theuser, of one of the markers at a particular position in one of the imagepanes, the system may automatically anchor a corresponding markerrepresenting a cross section of the three dimensional volume, that isorthogonal to the cross section represented by the user-placed marker,in each of the remaining views displaying an image slice including aregion corresponding to a portion of the three dimensional volume towhich the placed marker corresponds. Alternatively, a user-selectablecontrol, e.g., a button or icon, may be displayed, in response to whichselection, the system displays the corresponding markers in the otherviews according to the marker set by the user in one of the views.

In an example embodiment of the present invention, each of the panes102, 104, 106 may include a control 206, 208, 210, as described abovewith respect to FIG. 2, for scrolling through image slices of thecorresponding pane. If the user places a marker, e.g., the AC marker402, in a displayed axial image slice in one of the panes, the systemmay automatically place a corresponding marker, e.g., another AC marker402, in each of the displayed sagittal and coronal panes at a positionthat corresponds to the displayed axial slice in which the user droppedthe AC marker 402, as long as the images displayed in the sagittal andcoronal panes 104, 106 includes a region corresponding to the axialregion in which the user placed the AC marker 402.

Referring, for example, to FIG. 4, if the user places the PC marker 404at the position shown in axial pane 102, the system may responsivelydisplay the PC marker 404 in each of the coronal and sagittal panes 104and 106 shown in FIG. 4, because the image slices displayed in panes 104and 106 are cross sections of the axial slice displayed in pane 102,which cut through the anchored PC marker 404. On the other hand, if theuser places the AC marker 402 at the position shown in axial pane 102,the system may responsively display the AC marker 402 in the sagittalpanes 106 shown in FIG. 4, because the image slice displayed in pane 106is a cross section of the axial slice displayed in pane 102, which cutsthrough the anchored AC marker 402; but may omit responsive display ofthe AC marker 402 in the coronal pane 104 shown in FIG. 4, because theimage slice displayed in pane 104 is a cross section of the axial slicedisplayed in pane 102, which does not cut through the anchored AC marker402.

While the above has been described with respect to interaction by theuser with the axial pane 102, the same would apply if the user insteadinteracted with either of the panes 104, 106 for placement of the ACmarker 402 or PC marker 404 therein, in which case the system wouldcorrespondingly update the remaining two panes to include acorresponding marker, where appropriate.

Similarly, in response to a user shift of either of the markers to theright or left in the axial pane 102, the system may correspondinglyshift the corresponding marker in the coronal view to the right or left.The position of the corresponding marker within the sagittal pane 106may be left unchanged, although it may be removed from view or may comeinto view in the sagittal pane 106 in response to the shift of themarker in the axial pane 102, depending on whether the new positioncorrespond to a region represented in the displayed image slice of thesagittal pane 106. The same would be true if the user shifted the markerto the right or left in the coronal pane 104. Alternatively, in responseto the shift of the marker in the axial or coronal panes 102, 104, thesystem may scroll through slices in the sagittal pane 106 so that aslice including a region corresponding to the new position of the markeris displayed.

Similarly, if the user shifts a marker anteriorly or posteriorly in thesagittal pane 106, the system may correspondingly remove from view orbring into view the corresponding marker in the coronal pane 104according to the first described embodiment or may correspondinglyscroll image slices in the coronal pane 104 according to the secondembodiment. Similarly, if the user shifts a marker superiorly orinferiorly in the sagittal pane 106, the system may correspondinglyremove from view or bring into view the corresponding marker in theaxial pane 102 according to the first described embodiment or maycorrespondingly scroll image slices in the axial pane 102 according tothe second embodiment. Similar steps would be performed when shiftingmarkers in any of the panes, to correspondingly modify the remaining twopanes.

As noted above, according to an example embodiment, if the user shifts amarker in one of the panes from a first position to a second position,the first position having a corresponding position in a displayed imageslice in another of the panes, but the second position not having acorresponding position in the same image slice of the other pane inwhich the position corresponding to the first position is located, thesystem may scroll the image slices in the second view to one in which aposition corresponding to the second position is located.

For example, if, in the axial pane 102, the user shifts the AC marker402 to an image slice of a more anterior portion of the brain, thesystem may accordingly scroll through image slices of the coronal pane104 to one corresponding to a cross section at the more anterior portionof the brain and may display the AC marker 402 in the new position inthe newly displayed coronal image slice. The same would be true of thePC marker 404 if the user shifted the PC marker 404 in one of the views.However, while the scrolling may be performed so that the marker beingshifted is in view in each of the panes 102, 104, 106, the scrolling maycause the non-manipulated marker to move out of view in one of thepanes.

The system may calculate the distance between the points of the brainmarked by the user placed AC and PC markers, and may display thecalculated distance. For example, FIG. 4 shows a displayed AC-PCdistance identifier 410. As the user shifts the markers, the system mayupdate the displayed distance.

After placement of a marker, the user may directly scroll to other imageslices of any of the views, e.g., using the slider controls 206, 208,210. In each of the views, the system may display the markers with thegreatest brightness in the respective slices in which the markers wereanchored. As the user scrolls to other slices in which the marker wasnot anchored, the marker may be gradually dimmed until it is no longervisible. Therefore, the user may be able to determine from thebrightness of the markers, with respect to each of the panes 102, 104,106 whether the AC/PC is in the displayed slice, in a slice that is nearto the displayed slice, or in a slice that is not near to the displayedslice.

In an example embodiment of the present invention, the system maydisplay on the slice scroll bar markers that correspond to the AC and/orPC, at positions of the slice scroll bar that correspond to the imageslices to which the user-placed AC/PC markers have been anchored. Forexample, where an orange user-placed marker is used for anchoring theposition of the AC, the system may display an orange, e.g., vertical,line at a location of the slice scroll bar corresponding to the imageslice to which the AC has been anchored. Similarly, where a blueuser-placed marker is used for anchoring the position of the PC, thesystem may display a blue, e.g., vertical, line at a location of theslice scroll bar corresponding to the image slice to which the PC hasbeen anchored.

In an example embodiment of the present invention, the system maydisplay selectable buttons or icon corresponding to the AC and to thePC, in response to selection of which, the system scrolls to the imageslice corresponding to the selected button or icon in the active paneor, alternatively, in all of the panes.

The system may provide a zoom tool for carefully analyzing the AC/PCposition as set by the placement of the AC/PC marker. In an exampleembodiment, the zoom tool may also be used for fine tuning the placementof the marker. In an example embodiment, the user interface may includea graphical input object, such as a button in a toolbar, for opening thezoom tool. For example, in response to selection of the zoom toolbutton, the system may open the zoom tool in association with the lastactive one of the AC marker 402 and PC marker 404. Alternatively oradditionally, the system may be configured to display the zoom tool inresponse to a predefined type of interaction with, e.g., a double-clickof, either of the markers. For example, if the user double-clicks the ACmarker 402, e.g., using any suitably appropriate input device, thesystem may responsively display the zoom tool in association with the ACmarker 402. Similarly, if the user double-clicks the PC marker 404, thesystem may responsively display the zoom tool in association with the PCmaker 404.

For example, when the zoom tool is selected in one of the panes, thesystem may zoom in the portion of the image of that pane at which themarker is placed. For example, the marker may be displayed over aportion of an image of the brain. Upon selection of the zoom tool, aportion of the brain image in a region at which the marker was placedmay be zoomed in, while a remaining portion of the image remains at theprior zoom setting. The region zoomed may be a predetermined areameasured by pixel number, extending about a center of the AC/PC marker.For example, the AC/PC markers, when in a non-zoomed mode, may cover anarea of 1,809² pixels (a radius of 24 pixels). When the zoom tool isselected, the system may be configured to zoom in on an area of 7,854²pixels (a radius of 50 pixels) centered about the center of the initialarea of 1,809² pixels, so that those are displayed over 31,400² pixels(a radius of 100 pixels). It is noted that the anatomical region coveredby the initially placed marker may be smaller or larger than theanatomical region covered in a magnification window of the zoom tool. Inan example embodiment, in response to selection of the zoom tool in oneof the panes, the size of the marker shown in the remaining panesincreases or decreases to indicate the anatomical region covered by thethree-dimensional volume of the marker in the magnification window ofthe pane in which the zoom tool was selected.

FIG. 5 shown a screen shot 500 according to an example embodiment of thepresent invention, in which a magnification window 502 is displayed incoronal pane 104 after selection of the zoom tool while the coronal panewas active, e.g., by double-click of the AC marker 402 in the coronalpane 104. While the a region of the image is magnified in themagnification window 502, the remaining portions of the image remain atthe prior zoom setting, with the magnification window 502 overlaying theremaining portions of the image, i.e., being positioned within some ofthe remaining portions of the image while obstructing others of theremaining portions of the image.

While the magnification window 502 corresponding to the AC is displayedin coronal pane 104, large AC markers 505 are displayed in the axialpane 102 and sagittal pane 106, e.g., corresponding to the size of thethree dimensional volume represented by the magnified portion within themagnification window 502.

In an example embodiment, the zoom tool is selected by double-clickingthe AC/PC marker. In an example embodiment, after selection of the zoomtool, the portion of the image in the region at which the maker wasplaced may be returned to the zoom setting of the rest of the image byclicking in the pane or in any of the other panes at a location thatdoes not fall within the zoomed in region. According to an alternativeexample embodiment, a button, e.g., a GUI button, may be provided, inresponse to selection of which the system removes the magnificationwindow 502 and the corresponding markers in the other panes, and returnsto display of the AC/PC markers that were displayed prior to display ofthe magnification window 502.

The zoom control may further include sub-controls for increasing ordecreasing the zoom of the zoomed in region by predetermined amounts.For example, a ‘+’ may be displayed which is selectable for increasingzoom and a ‘−’ may be displayed which is selectable for decreasing zoom.For example, the zoom tool shown in FIG. 5 includes a zoom increasecontrol 510 displaying ‘+’ to indicate its use for increasing zoom, andis displayed joined to the magnification window 502 at the top of themagnification window 502. The zoom tool shown in FIG. 5 also includes azoom decrease control 512 displaying ‘−’ to indicate its use fordecreasing zoom, and is displayed joined to the magnification window 502at the bottom of the magnification window 512. Each selection of thezoom increase control 510 or zoom decrease control 512 causes the systemto respond by respectively increasing or decreasing the magnification.In an example embodiment, the system may be configured with a limit forzoom increase and/or decrease.

In an example embodiment of the present invention, in response to eachoperation of the zoom increase control 510 and in response to eachoperation of the zoom decrease control 512, the system maycorrespondingly modify the size of the corresponding markers 505 in theother two panes to reflect the modification of the region reflected inthe magnification window 502. Alternatively, the system may beconfigured to leave the markers 505 at their original size set inresponse to the initial activation of the zoom tool.

While the portion of the image in the region corresponding to the markeris zoomed, the user may click and drag the magnification window 502, inresponse to which the magnification window 502 and the placement of theanchoring of the corresponding anatomic object (AC or PC) may becorrespondingly shifted. Thus, the user is able to shift the recordedplacement of the AC or PC while the region is zoomed, which may help theuser select appropriate placement of the marker. After deselecting thezoom tool, the corresponding AC marker 402 or PC marker 404 having thesame zoom setting as the remaining portions of the image may bedisplayed at the new location set by the shift of the magnificationwindow 502.

As noted above, a slider control 206, 208, 210 may be operated forscrolling through slices of a respective one of the panes 102, 104, 106.As described above, in an example embodiment, in response to suchscrolling in a mode in which the AC marker 402 and PC marker 404 may beset and are displayable (assuming the image slice to which they areanchored is displayed), the AC marker 402 and/or PC marker 404 may fadein and out of display. In an example embodiment, although scrolling froma first image slice to a second image slice may cause the AC marker 402and/or PC marker 404 to fade from display, if the user operates an imageslice scrolling control, e.g., the slider control 206, 208, or 210, toscroll through image slices of a pane while the magnification window 502is displayed, the system moves the magnification window 502 and theanchoring of the respective anatomical object to the image slice towhich the user scrolls. Alternatively or additionally, further controlsmay be attached to the magnification window 502 for scrolling. Forexample, buttons similar to controls 510 and 512 may be displayed, whereselection of a first one of the buttons causes scrolling of the slicesin one direction and selection of a second one of the buttons causesscrolling of the slices in the opposite direction.

Thus, the magnification window 502 may remain in view in a singledisplay position, while the image slices being displayed are scrolled.On the other hand, as described above, if the user operates the imagescroll control to scroll through the image slices while the markerregion is not zoomed, the anchoring of the anatomical component (AC orPC) of the last-active one of the markers (AC marker 402 or PC marker404) remains in the prior image slice, and the image slices are scrolledwhile the corresponding marker fades away with increasing distancebetween the scrolled-to image slice and the slice in which thecorresponding marker was set.

Thus, according to an example embodiment of the present invention, thesystem may provide two distinct methods by which to shift the AC marker402 or PC marker 404 between image slices. According to a first method,the system may scroll to different image slices in one of the panes 102,104, 106 while the magnification window 502 corresponding to therelevant one of the markers is displayed in the respective pane in whichthe image slice scrolling control is operated. According to a secondmethod, the user may shift the marker in one of the panes to a differentslice by shifting placement of the relevant marker in another one of thepanes, when the magnification window 502 is not displayed. For example,if the user shifts the AC marker 402 superiorly or inferiorly in thecoronal pane 104 or sagittal pane 106, the system may shift theanchoring of the AC (and the display of the AC marker 402) to adifferent axial image slice. Similarly, an anterior or posterior shiftin the axial pane 102 or sagittal pane 106 causes a shift to a differentcoronal slice. Similarly, a shift to the right or left in the axial pane102 or coronal pane 104 causes a shift to a different sagittal slice.

The magnification window 502 may be round as shown in FIG. 5, but may beany other suitable shape in alternative embodiments. As shown in FIG. 5,in an example embodiment, the magnification window 502 may includecross-hairs via which to easily identify the portion of the image thatlies at the center of the marker, to further help the user correctlyplace the marker.

In an example embodiment of the present invention, the system mayexecute an image enhancement algorithm on a zoomed in region. Forexample, the image enhancement may be applied to the region displayedwithin the magnification window 502. The image enhancement may includeapplying one or more image filters to sharpen edges, to facilitateidentification of boundaries and structures. In an example embodiment,the image enhancement may be performed automatically, in response toopening of the magnification window 502 and/or for further zoomingwithin the magnification window 502. Alternatively or additionally, oneor more image processing filters may be user selectable for applicationto the zoomed in region. In an example embodiment, the system mayprovide a list of selectable image filters for application to the zoomedin region. For example, a selectable menu option may be provided whenthe magnification window 502 is open, for opening the list of selectableimage filters. In an example embodiment, the system is configured forinput of user-generated custom filters, which may then be added to sucha list for application to the region within the magnification window502.

In an example embodiment, the system may output a patient atlas based onthe identified AC/PC, as described below. Additionally or alternatively,the system may register the patient's MR image, in which the AC and PCare identified, to another volume, such as a DTI atlas or an MR image ofanother patient.

After placement of a marker or a shift of the position of the marker,the user may select an “undo” control, in response to which the systemmay undo the last placement or shift to and reposition the marker at aprior placement. A complete shift that is separately undoable may be anaction including a selection, a drag, and a drop, such that theselection of the “undo” control may cause the processor to repositionthe marker at the position at which the marker was selected prior to thedrag. For repeated selections of the undo control, the system may undo aseries of changes to the marker placement. Similarly, after operation ofthe pivot and stem tool 110, to modify the relative positions ofdifferent images, the user may select the “undo” control, in response towhich the system may undo the last modification of the relativepositions. For repeated selections of the undo control, the system mayundo a series of changes to the relative positions. The “Undo”functionality may be provided for marker placements, leadwireplacements, and/or alignment and/or scaling of images.

Auto Histogram and Level of CT+MR in Different Screens

In an example embodiment of the present invention, the system mayprovide for auto-correction of images in order to provide a best view ofrelevant features in the images. The view of the features may be usefulfor a user to properly co-register images based on the positions of thefeatures, to verify a previously performed co-registration, to selectand/or verify lead tip and shaft placement, to select and/or verify MCP,MSP, AC, PC or other landmark points, and/or to determine how to setstimulation parameters in a stimulation programming environment.

In an example embodiment, the system may implement the auto-correctionprior to initial display of the images. Alternatively, the system mayinitially display the images without the correction and perform theauto-correction in response to a user input instruction to do so. Forexample, the system may display a graphical button selectable by theuser for instructing the system to perform the auto-correction.

In an example embodiment of the present invention, the type ofauto-correction performed by the system depends on the imaging modalityof the image. For example, the system may auto-correct an MR image usinga first imaging correction method and auto-correct a CT image using asecond, different, correction method. The different methods may beimplemented automatically without user instruction, or may beimplemented in response to a single instruction to auto-correct overlaidimages.

For example, in response to user operation of a user interface control,the system automatically adjusts greyscale values of the pixels for abest image.

In an example embodiment of the present invention, the system mayautomatically enhance an MR image by modifying a distribution of pixelvalues assigned to the pixels of the image. For example, referring toFIG. 6, the system may, at step 600, generate a histogram based on thegreyscale values of the image, which histogram plots the percentage ofpixels of the image at various greyscale values, e.g., beginning withwhite and ending with black, for example, such as histogram 700 shown inFIG. 7. At step 602, the system may identify a region of the graphcorresponding to the block of continuous color values having thegreatest percentage of pixels. Such a block will usually form a curve,such as curve 705 in block 702 of FIG. 7. The identification of theblock may include determining the pixel values at which the curve beginsand ends, which pixel values may be set as the outer limits of theblock. At step 604, the system may reassign the pixel values to spreadthat curve out over a greater number of pixel values, so that fewer ofthe pixels are confined to that range of pixel values than prior to thespread and a greater number of pixels are assigned the pixel values thatare external to that range than prior to the spread. Any suitablyappropriate histogram equalization method may be used for modifying theassignment of pixel values to the pixels of the MR image.

Such a modification of the MR image causes the MR image to be moreclearly show different anatomical features because a greater variety ofpixel values are used for showing the different anatomical features.

As noted above, the system may be configured to differently modify a CTimage. What is often relevant in the CT image with respect tostimulation programming is the skull and the leadwire(s). CT imagesbecomes saturated for matter that is as dense as bone or denser,saturation referring to where the CT image no longer differentiatesbetween structures of different density. The system may accordingly beconfigured to remove all pixel information for those pixels havingvalues other than the saturated level. This would mostly leave the skulland the leadwire(s). For example, FIG. 8 is a flowchart that shows anexample method the system may perform for auto-correction of a CT image.At step 800, the system may obtain a pixel value for one of the pixelsof the CT image. At step 802, the system may compare the obtained pixelvalue to a saturation threshold pixel value. If the pixel value does notmeet the threshold, indicating that it is not at the saturation level,the system may, at step 804, remove the pixel information. For example,the system may set the value of the pixel to black, e.g., 0. If thepixel value meets the threshold, indicating that it is at the saturationlevel, the system may skip step 804 for that respective pixel. If theimage includes additional pixels not yet so analyzed, the system mayreturn to step 800 for selection of another pixel of the CT image.Otherwise, the method may end.

Other significant structures may be ventricles and sulci, when verifyingfusion of and MR and CT. Thus, in an alternative example embodiment, thesystem is configured to automatically modify the levels of the CT imagesuch that the ventricles and sulci are clearly visible in the CT image.

Thus, the system may be configured to display a images overlaying eachother, where the images are auto-corrected via different auto-correctionmethods in response to the same auto-correct trigger.

Atlas Registration Identifying a Predefined Plane or Line

In an example embodiment of the present invention, the system isconfigured to output a graphical representation of a predefined line orplane, e.g., which may be registered to one or more images of ananatomical region. The predefined line or plane may be an anatomicallysignificant line or plane. In an example embodiment, the plane may bethe mid-sagittal plane (MSP), a theoretical plane dividing the left andright hemispheres of the brain. The system may store in memory thelocation of the predefined line or plane, e.g., the MSP, relative to athree-dimensional volume of each of one or more images of one or moreimaging modalities. The output representation of the line or plane,e.g., the MSP, may overlay and be relative to the display of one or moreof such images in accordance with the recorded relative position. Thefollowing discussion will refer to the MSP, but may be applied similarlyto other lines or planes.

The graphical representation may be output in a user-interactiveinterface via which the user may interact with the representation of theMSP and/or with a displayed image over which the MSP representationoverlies to modify the location of the MSP relative to the image. Theuser may interact with the user interface for translating and/orrotating the MSP representation and/or to translate and/or rotate theimage, e.g., an MR image, to correctly align the displayed MSPrepresentation with the MR, to coincide with the actual position of theMSP with respect to the anatomical elements as displayed in the MRimage.

For example, FIG. 9 shows an example screen shot 900 in which a brokenline is used as an MSP representation 902. The MSP representation 902 isdisplayed in each of the axial pane 102 and the coronal pane 104,indicating a cross-section of each of the respective image slices ofthose panes through which the MSP cuts. In an example embodiment, asshown in FIG. 9, the MSP representation 902 is omitted from sagittalpane 106 because the MSP does not cut through a cross-section of asagittal image slice. Instead, the MSP is theoretically in line with oneof the sagittal image slices, encompassing an entirety of that sagittalimage slice.

In an example embodiment of the present invention, and as shown in FIG.9, the pivot and stem tool 110 may be provided in each of the axial pane102 and the coronal pane 104 while in a MSP anchoring mode in which theuser may interact with the user interface for modifying a relativeposition of the MSP to the image. The pivot and stem tool 110 may beoperated for shifting and rotating respective ones of the imagesdisplayed in the axial pane 102 and coronal pane 104. It is noted thatin alternative example embodiments, other user input methodologies maybe used, for example, for all operations described herein as beingperformable via interaction with the pivot and stem tool 110. Forexample, a user may drag the image with a finger or stylus via a touchscreen, where a motion tracing an arc is interpreted as a rotation, orwhere two fingers simultaneously dragging in opposite directions isinterpreted as a rotation, and where other simpler left/right/up/downmotions are interpreted as translations. Alternatively, the user mayclick an image and then click directional keyboard buttons or GUIbuttons, such as straight and/or curved directional arrows, fortranslating and/or rotating the image.

Referring again to FIG. 9, in response to a left or right shift of theimage in the axial pane 102, thereby oppositely shifting the recordedlocation of the MSP (and the MSP representation 902) relative to theimage of the axial pane 102, the system may correspondingly shift theimage displayed in the coronal pane 104 to the left or right, therebyoppositely shifting the record location of the MSP (and the MSPrepresentation 902) relative to the image of the coronal pane 104.Similarly, in response to a left or right shift of the image in thecoronal pane 104, thereby oppositely shifting the recorded location ofthe MSP (and the MSP representation 902) relative to the image of theaxial pane 104, the system may correspondingly shift the image displayedin the axial pane 102 to the left or right, thereby oppositely shiftingthe record location of the MSP (and the MSP representation 902) relativeto the image of the axial pane 102.

In response to either the left or right shift of the image in either ofthe axial pane 102 or coronal pane 104, the system may automaticallyscroll the image slices in the sagittal pane 106 so that the displayedsagittal image slice is the cross-section of the axial and coronal imageslices through which the MSP representations 902 in each of the axialand coronal panes 102, 104 extends.

A left or right shift of the image in the sagittal pane 106 maysimilarly cause the system to modify the image displayed in the coronalpane 104, because the position of the image in the sagittal pane 106along the horizontal axis may define the center point of the brain alongthe line extending between the anterior and posterior extremities of thebrain, thereby redefining the origin from which coronal slice distancesare measured. If a displayed coronal image slice is indicated to be atthe origin (0.0 mm), then, in response to the left or right shift of theimage in the sagittal pane, a different image slice at the newly definedorigin may be displayed in the coronal plane 104.

Similarly, in response to a shift of the axial image upwards ordownwards, the system may modify the image of the coronal pane 104 toreflect the new origin coordinate with respect to the anterior andposterior directions.

Similarly, in response to a shift of the sagittal image slice upwards ordownwards, the system may change the image in the axial plane 102 toreflect the new origin coordinate with respect to the superior andinferior directions. The same modification of the axial image slice maybe performed in response to an upwards or downwards shift of the coronalimage.

A rotation of the image in the axial pane 102 redefines the coordinateof the most anterior and posterior points, and the most left and rightpoints of the image. Since the coronal pane 104 displays image slicesthat are orthogonal to a line extending between the anterior andposterior extremities and parallel to a line extending between the leftand right extremities, therefore, in response to the rotation of theimage in the axial pane 102, the system correspondingly changes theimage of the coronal pane 104 to be of a slice that is orthogonal to thenewly defined line extending between the newly defined anterior andposterior extremities.

Similarly, since the sagittal pane 106 displays image slices that areparallel to a line extending between the anterior and posteriorextremities and orthogonal to a line extending between the left andright extremities, therefore, in response to the rotation of the imagein the axial pane 102, the system correspondingly changes the image ofthe sagittal pane 106 to be of a slice that is parallel to the newlydefined line extending between the newly defined anterior and posteriorextremities.

Similarly, a rotation of the image in the coronal pane 104 redefines thecoordinates of the most superior and inferior extremities and left andright extremities. The images displayed in the axial and sagittal panes102, 106 may therefore be correspondingly changed. Similarly, a rotationof the image in the sagittal pane 106 redefines the coordinates of themost superior and inferior extremities and the most anterior andposterior extremities. The images displayed in the axial and coronalpanes 102, 104 may therefore be correspondingly changed.

The above discussion regarding responding to translational or rotationalchanges to an image in one of the panes with modifications of imageslices in other panes applies as well to other modes described herein inwhich images are translatable and/or rotatable, e.g., a mode in whichthe alignment of an MR image and a CT image is modifiable and/or a modein which the AC marker 402 and PC marker 404 can be set.

In an alternative example embodiment, the system is configured forreceiving user input for selecting three points in MR slices fordefinition of a plane with respect to an image, e.g., the MR, that isrepresentative of the MSP. For defining the MSP with respect to allviews (axial, coronal, and sagittal) it may be required for the systemto receive the input of at least two of the points in different axialslices. This does not require that the selection be within different MRaxial slices. Instead, for example, two of the points may be selected inan axial MR image, and a third may be selected in a coronal MR image,where the third point is in a different anatomic axial slice than thethat to which the MR axial image slice, in which the first two pointswere selected, corresponds. It may further be required for the system toreceive the input of at least two of the points in different coronalslices. This does not require that the selection be within different MRcoronal slices. Instead, for example, two of the points may be selectedin an axial MR image, and a third may be selected in a coronal MR image,where the third point is in a different anatomic coronal slice than thatto which the MR axial image slice, in which the first two points wereselected, corresponds. (Alternatively, two of the points may be selectedin the coronal view and the third point may be selected in an axial viewat a point that corresponds to a coronal slice different than the one towhich the coronal image, in which the first two points were selected,corresponds.)

It is noted that the MSP does not always correspond entirely to a singleimage slice displayed in the sagittal pane 106. Therefore, the threepoints cannot be noted entirely in the sagittal view in most instances.Additionally, because the MSP may be angled in two orthogonal directionswith respect to a plane corresponding to a sagittal image slice,therefore it may be more practical to select the points in the axial andcoronal panes 102, 104 rather than the sagittal pane 106.

In an example embodiment, the system may be configured to provide a userinterface where user interaction for defining the MSP is limited to theaxial and coronal images, e.g., by translation or rotation of arepresentation of the MSP or of the images, or by placement of the threepoints in the images, while locking the sagittal image against suchinteraction, because of the greater preciseness expected in mostinstances in defining the MSP in the axial and coronal panes 102, 104due to common angling of the MSP in the directions orthogonal to thesagittal image.

Referring to the definition of the MSP by placement of three points, itmay be required to receive input of three points in different axial andcoronal slices because, otherwise, only a single line would have beendefined, and how the plane extends from the defined line to planesparallel to the one in which the line was defined would be unknown. Forexample, FIG. 10 shows three points 1000, 1010, and 1020 defined along asingle line 1050 within a three dimensional space. The line 1050 isincluded in each of planes 1070 and 1080. However, the planes 1070 and1080 cross the line 1050 at different angles. Therefore, definition ofthree points along a single line does not provide enough information todefine a plane. For example, if the three dimensional volume of FIG. 10would represent a brain, and the points 1000, 1010, and 1020 would bedefined along a single line in a sagittal image slice, the angle of theMSP plane to the plane defined by the sagittal slice would be unknown.

In an example embodiment, the system may initially receive inputidentifying the location of the AC and the location of the PC, e.g., asdescribed in detail above, which the system may use as two of the pointsfor defining the MSP. Subsequent to receipt of the AC and PC locations,the system may present a UI in which to select a third point forcompletion of the definition of the MSP. For example, the user may setthe AC and PC points in an axial image slice and set the third point ofthe MSP in a coronal image slice.

In an example embodiment of the present invention, the system mayprovide two or more, e.g., all, of the described functionality foraligning multiple images to each other, setting the AC and PC locations,identifying the MSP, modifying a position and/or size of a flashlightbar, and scrolling image slices, in a single GUI presentation. Inexample embodiments, the system may be configured not to combine variousones of those described features in a single GUI. In an exampleembodiment, the system may be configured to present certain of thedescribed features in separate GUIs in a defined sequence, which may befollowed by inputting “Next” and “Back” commands. For example, thesystem may be configured to initially present a GUI via which a user mayco-align different images. In response to an instruction to proceed to anext step, the system may present a GUI via which to set the MSP. Inresponse to yet another instruction to proceed to a next step, thesystem may be configured to present a GUI via which to set the AC and PCpositions. In an example embodiment, user interaction with any of theGUIs presented in sequence may be recorded and may accordingly impactthe other GUIs presented at different points in the sequence, regardlessof whether the other GUIs are displayed in response to a “Next”instruction or a “Back” instruction.

In an example embodiment, a progress bar may be provided that visuallyindicates a current location with respect to a sequence of steps to beperformed, where the current location indicates the present step beingperformed. Each such step represented by the progress bar may refer to aparticular user interface, where one or more, e.g., each, of suchinterfaces provide functionality for performing a number of sub-steps.Further, a first progress bar may be provided showing a location withrespect to high-level steps, and a further progress bar may be providedshowing a location with respect to low-level level steps within one ofthe high-level steps. For example, a first progress bar may show thesteps of inputting patient information, e.g., by import of patientimages and/or records, image/atlas registration, and programming. Withinregistration, a progress bar may show the steps of fusing MR and CTscans (or, in other embodiments, other images), selection of the MSP,selection of AC/PC, location of the lead tip(s), and location of thelead shaft(s).

With respect to import of images, the images may be imported from asystem in which the images are associated with a patient record. Theregistration and/or programming system may include an import option forselecting an image from such an external system. In response toselection of the image from the external system, the system mayautomatically create a new patient file in the registration and/orprogramming system based on information of the patient record of theexternal system that is associated with the selected image. For example,any one or more of the patient name, age, gender, DOB, address,diagnosis, etc. may be imported and may automatically populate fields ofan electronic record of the registration and/or programming system.

In an example embodiment of the present invention, the system mayprovide a patient listing, and for each listed patient, may indicate anactive stage. The listing may be updated in response to import ofinformation from an external system, as described above. With respect tothe active stage, for example, if the image/atlas registration UIscreens have not yet been completely traversed for setting theregistration for a patient, the system may list that patient in apatient grouping under the heading “registration,” while the system maylist a patient for whom the registration has been completed in a patientgrouping under the heading “programming.” Moreover, the system maydisplay for one or more, e.g., each, of the listed patients a statusindicator showing a degree to which the stage has been completed. Forexample, for a patient for whom 75% of the registration steps have beenperformed, the system may display a circle, 75% of the perimeter ofwhich is highlighted, or that includes a line or arrow extending fromthe interior of the circle towards a point on the perimeter of thecircle corresponding to 75% of the circle, e.g., where thetop-dead-center of the circle represents 0% and a clockwise rotation isassumed. Alternatively other markings relative to a circle or othergeometric shape may indicate the completion percentage, e.g., thepercentage of the shape that is filled may provide the indication.

Atlas Scaling

As noted above, image alignment and definition of the AC and PClocations may be used for registration between anatomical volumes of apatient and other defined volumes, e.g., of an anatomical atlas.Definition of the MSP may similarly define the line extending betweenthe anterior and posterior extremities, the line extending between thesuperior and inferior extremities, and the line extending between theright and left extremities, which information may be useful foranisotropic scaling, described in detail below. The defined MSP may alsobe used for proper alignment of a three-dimensional volume such as thatof another patient from a patient population or of an anatomical atlaswith the patient image, according to which proper alignment, thenon-patient volume may be registered to the patient's anatomy asreflected by the patient image.

For creating a patient-specific atlas, the system may provide a userinterface via which a user may interact for initially lining up thepatient MR to the atlas. This may be performed, for example, in the samemanner as that described above with respect to aligning MR and CTimages.

Alternatively, the system may additionally provide a user interface viawhich the user may additionally identify the mid-commissural point (MCP)in the patient image. The system may automatically align a patientimage, e.g., an MR image, with the atlas subsequent to, and inaccordance with, user input of the location of the MSP, AC/PC, and/orMCP in the patient image.

For example, the system may line up the AC/PC lines and/or MSP of theatlas and the patient MR, and may line up the MCP identified in thepatient MR with the MCP of the atlas. Alternatively, the system mayinitially line up the atlas and the patient MR based on the AC/PC lineand/or the MSP, and the user may then interact with a user interfaceprovided by the system to shift one of the atlas and the patient MRrelative to the other, to line up the point of the patient MR identifiedby the user as corresponding to the MCP with the MCP of the atlas.

The MCP is the mid-point between the AC and PC. Therefore, in an exampleembodiment of the present invention, subsequent to, for example,user-identification of the AC and PC, the system may automaticallyidentify and record the MCP as the mid-point therebetween, user inputnot being required for such identification.

The distances between the respective AC/PC of each of the atlas and thepatient MR can differ. The patient's MR image may therefore be requiredto be scaled relative to the atlas. In an example embodiment of thepresent invention, the system may scale the atlas (or other non-patientvolume, e.g., from a patient population) automatically.

FIG. 11 is a flowchart that shows example steps the system may performto automatically align and scale an atlas (or other non-patient) volumeto the patient volumes represented in the patient image. At step 1100,the system may obtain the definition of the AC and PC, e.g., via userinteraction with a user interface as described above. At step 1102 b,the system may perform steps 1105 and 1112, e.g., in accordance with theAC/PC definitions obtained in step 1100. At step 1105, the system maydetermine the MCP as the mid-point between the AC and PC. Step 1105 mayalso be a part of a step 1102 a on which basis the system may performstep 1115 described below. Step 1102 a may also include step 1110. Atstep 1110, the system may obtain the definition of the MSP, e.g., viauser interaction with a user interface as described above.

On the basis of step 1102 a, including the determined MCP and obtainedMSP, the system may perform step 1115. At step 1115, the system maythree-dimensionally align the atlas (or other non-patient) and patientvolumes.

Step 1115 may be part of step 1111. Step 1112, which is a part of step1102 b, may also be a part of step 1111. At step 1112, the system maycalculate the distance between the AC and the PC based on the AC/PCdefinition obtained at step 1100. The calculation may be based on aknown relationship, with which the system is programmed, betweenanatomical area and image pixels at various resolutions, and thedistance in the image between the points to which the AC and PC wereanchored.

Based on step 111, in which the volumes are aligned and the AC-PCdistance is determined, the system may, at step 1120 scale the atlas (orother non-patient volumes) to approximately match the patient volumes.

The following are four example methods that may be used for performingthe scaling.

In an example embodiment, the atlas is scaled linearly (by the sameamount at all distances from the MCP) and isotropically (by the sameamount in all directions).

In an example embodiment, the atlas is scaled linearly andanisotropically (by different amounts in different directions). In thisregard, the inventors have discovered that an anisotropic scaling of theatlas would usually result in a better fit to patient volumes than anisotropic scaling.

In an example embodiment, the atlas is scaled non-linearly (by differentamounts at different distances from the MCP) and isotropically. In thisregard, the inventors have discovered that a non-linear scaling of theatlas would usually result in a better fit to patient volumes than alinear scaling.

In an example embodiment, the atlas is scaled non-linearly andanisotropically.

Referring to the linear and isotropic scaling, in an example embodiment,the atlas may be stretched or contracted equally in all directionsincluding right, left, the anterior direction, the posterior direction,the inferior direction, and the superior direction, with the MCPremaining in place, until the distance between the AC and PC of theatlas equals the distance between the AC and PC of the MR image.

However, it has been determined that, while the above method may providea rough atlas of the patient brain, it is often advantageous to scalethe atlas anisotropically. For example, it is often advantageous toscale the atlas in the anterior and posterior directions, i.e., in adirection approximately parallel to the line connecting the AC and PC,to a greater extent than in other directions. Therefore, in an exampleembodiment, the scaling in the superior and inferior directions and tothe left and the right may be, for example, to approximately 0.8 theamount by which the atlas is scaled in the anterior and posteriordirections. For example, as shown in FIG. 12, the anisotropic scalingmay be by the following factors:

${a = \frac{{AP}_{MR}}{{AP}_{A}}};$

LR_(MR)=(0.8*a+0.2)*LR_(A); and DV_(MR)=(0.8*a+0.2)*DV_(A), where AP_(X)is the distance in the anterior-posterior directions, LR_(X) is thedistance in the left-right directions, DV_(X) is the distance in thesuperior-inferior directions, X_(MR) is the distance in the patient's MRspace, and X_(A) is the distance in the original atlas space. The 0.8factor was determined by examining the respective ratios of the lengthof the brain in each of anterior/posterior, left/right, andsuperior/inferior directions to the distance between the AC and PC in anumber of patients, and then examining the ratio of the left/right ratioto the anterior/posterior ratio and the ratio of the superior/inferiorratio to the anterior/posterior ratio. In nine studied patients, it wasfound that the ratio of the anterior/posterior distance to the AC-PCdistance was an average±standard deviation of 6.07±0.66; the ratio ofthe left/right distance to the AC-PC distance was an average±standarddeviation of 4.85±0.52; and the ratio of the superior/inferior distanceto the AC-PC distance was an average±standard deviation of 4.84±0.42.Therefore, the ratio of the left/right ratio to the anterior/posteriorratio was 4.85/6.07=˜0.8, and the ratio of the superior/inferior ratioto the anterior/posterior ratio was 4.84/6.07=˜0.8.

The 0.2 offset may be applied because of the possibility that thedistance between the AC and PC in the atlas and in the patient MR areequal or substantially equal, such that no scaling is required in theanterior/posterior direction, in which case the 0.2 offset would providethat no scaling is performed in the other directions as well.

In an example embodiment of the present invention, the system may beupdated over time with information regarding ratios of the lengths ofthe brain in the different directions for different patients, and thesystem may recalculate the factor by which to perform theanisotropically scaling in accordance with such updated information.

As noted above, it has also been determined that it may be beneficial tonon-linearly scale the atlas. For example, the system may scale theatlas by a lesser degree at greater distances from the MCP than atsmaller distances (or vice versa). Further, whether to linearly ornon-linearly scale the atlas may depend on the direction. For example,the atlas may be linearly scaled in the anterior and posterior directionand non-linearly scaled in the other directions, where the scale amountis inversely proportionate to the distance from the MCP, i.e., thegreater the distance, the less the scale factor.

In an example embodiment, the system may be programmed with a predefinedfunction to scale the atlas as a function of the difference between theAC-PC distance of the atlas and that of the patient and as a function ofdistance between the coordinate of the atlas being scaled and the MCP.In an alternative example embodiment a different anatomical landmarkother than the MCP may be selected as that to which a distance ismeasured and used for determining the degree of scaling.

In an example, the system initially scales the atlas linearly, e.g., asa function of a difference between the AC-PC distance of the atlas andthat of the patient, and provides a user interface by which the user mayprovide input for modifying the atlas with a non-linear scaling, forexample, using a user interface as described in, e.g., the '330, '312,'340, '343, and '314 applications concerning FIGS. 9 and 10 of the '330,'312, '340, '343, and '314 applications.

While the above has been described with respect to the AC, PC, MSP, andMCP, it is noted that the initial alignment of the patient MR and theatlas may be performed using other landmarks, and while the centering,stationary registered point about which the scaling is performed hasbeen described above as the MCP, it is noted that other landmarks may beused as the stationary registered point about which the scaling isperformed. For example, other landmarks may be used where the mostrelevant anatomical regions are of different portions of the brain inwhich the AC, PC, MCP, and/or MSP are not the most relevant part, orwhere the focus of the stimulation is an anatomical region other thanthe brain.

In an example embodiment, the system may be preconfigured with settingsfor automatically scaling the atlas in the described ways and/or thedescribed amounts. In an example embodiment, the system may provide auser interface for inputting scaling values on a per direction basis,and/or for inputting a scaling factor as a function of distance, e.g.,on a per direction basis.

While the above methods for scaling of the atlas have been describedwith respect to an MR image, the methods may be similarly applied toother imaging modalities. For example, after a CT is registered to an MRand the AC and PC registered in the CT, the atlas may be lined up withthe CT and the scaling of the atlas may be to the CT image.

Automated Atlas Registration

Rigid, Affine, and B-Spline Registration with or without Skull Stripping

In an example embodiment of the present invention, a patient atlas maybe automatically generated without use of identified points, e.g., AC,PC, MCP, MSP, within the patient image. According to this method, stepsfor identification by the user of the MSP, AC, and PC, as describedabove, may be omitted.

The system may store a plurality of MRs of a patient population andcorresponding atlases generated for those MRs. In an example embodiment,the system may further store an atlas that is an average of theindividual atlases corresponding to the MRs of the patient population.In an example embodiment, the system may further store, for each of aplurality of subsets of the patient population MRs, a respective averageatlas. The subsets may be formed by grouping MRs by one or more metricsselected from a group of metrics including patient condition, such asdisease indications and/or injuries; patient age; patient sex; patientheight; patient weight; overall brain size; target VOA; and/or MR scantype (e.g., T1 or T2).

The system may select one of the stored atlases for automaticregistration to the patient MR. The selection may be based on comparisonof one or more metrics selected from a group of metrics includingpatient condition, such as disease indications and/or injuries; patientage; patient sex; patient height; patient weight; overall brain size;target VOA; and/or MR scan type (e.g., T1 or T2).

According to an embodiment in which target volumes are used forselection of patient population MR(s), if there are multiple targetvolumes, the system may use an average or a weighted average of thevolumes. Which target volumes to use as the basis for the selectionand/or the weights of the various volumes may be manually selected ormay be automatically selected, e.g., based on importance of therespective target volumes for treating the patient's disease state.

The selection may alternatively or additionally be based on MER data.For example, the atlas corresponding to the stored MR image associatedwith MER data most closely matching that of the patient may be selected.The MER data may be a factor considered in addition to a result of amutual information algorithm which determines a similarity betweenimages. For example a function may be used to determine similarity whichweights different factors, such as MER data and mutual information.

In an example embodiment, the system may select a subset of the patientpopulation based on factors delineated above, and then select theaverage atlas of that subset.

In an example embodiment, the system may always select the average atlasof all of the patient population.

In an alternative example embodiment, an average atlas, e.g., that ofall of the patient population or that of a particular subset of thepatient population may be selected only if no single stored MR image ofthe patient population is determined to be sufficiently similar to thatof the patient's MR image and/or if the MER data and/or the other usedfactors are not sufficiently similar.

The system may then warp the selected patient population MR, to which anatlas has been registered, to the patient MR image to obtain apatient-specific atlas, using one or more image registration processes.A mutual information algorithm may then be used to determine how wellthe atlas has been modified by the registration processes.

In an example embodiment, the system may register the selected atlas tothe patient MR image using a rigid registration. The rigid registrationincludes a rotation and/or a translation of the atlas, but does notinclude scaling.

In an alternative example embodiment, the system may perform atransformation method which the inventors have discovered produces amore precise atlas registration. The method includes performing a rigidregistration followed by an affine registration of the patientpopulation MR. The affine registration may include a modification of thetype x→Ax+b, where Ax is a linear transformation, and +b refers to atranslation, and/or may include a translation, a rotation, scaling,and/or shear tranforms. The affine registration may include a non-linearand/or anisotropic modification of the patient population MR, e.g.,where the non-linearity and anisotropy is as described above. The rigidregistration may be initially performed to provide a better startingpoint for performance of the more complex affine registration, providingfor a faster and more accurate registration than performance of just theaffine registration. The inventors have discovered that performance ofthe affine registration following the rigid registration usuallyprovides a more accurate patient atlas than where only the rigidregistration is performed.

In an alternative example embodiment, the system may perform atransformation method which the inventors have discovered produces aneven more precise atlas registration. The method includes performing arigid registration, followed by an affine registration, as describedabove, followed by a B-spline registration. Alternatively, the B-splineregistration may be performed prior to the affine registration. TheB-spline registration may include a non-linear and/or anisotropicmodification of the patient population MR, e.g., where the non-linearityand anisotropy is as described above. The affine registration may act onthe patient population image and the current patient image as a whole,whereas the B-spline registration may act on smaller sub-volumes of theimages. The rigid registration may be initially performed to provide abetter starting point for performance of the more complex affine andB-spline registrations, providing for a faster and more accurateregistration than performance of just the affine and/or B-splineregistration. Additionally, there is a chance that a B-splineregistration algorithm performed directly on the original patientpopulation image would fail because a B-spline registration algorithmmight not be able to resolve images that are too dissimilar.

In an alternative example embodiment, the system may perform a variantof the above-described registrations which the inventors have discoveredproduces an even more precise atlas registration. The method includesinitially removing portions of the image corresponding to the patient'sskull, and then performing one of the above-described registrationmethods, i.e., rigid, rigid+affine, or rigid+affine+B-spline. The skullmay be irrelevant to the registration process. Removal of the skull datawould allow the algorithms to base the transformation on a greaterpercentage of relevant information.

While it is advantageous to perform a combination of skull stripping,rigid registration, affine registration, and B-spline registration forobtaining a precise registration, in an example embodiment it may beadvantageous to omit one, some, or all of the skull stripping, affineregistration, and B-spline registration steps, to reduce processingload, e.g., depending on processing capacity of the device being used.

Accordingly, referring to FIG. 13, in an example embodiment of thepresent invention, the system may, at step 1300, remove image data fromthe patient's MR image determined to represent the skull. At step 1302,which may be performed, for example, subsequent to step 1300 in anexample embodiment, the system may select a patient population imagefrom a repository of patient population images to which respectiveatlases have been registered. At step 1304, the system may perform arigid registration to warp the selected patient population image, e.g.,MR, to fit the skull-stripped patient image. At step 1306, the systemmay subsequently perform an affine registration of the thus far warpedpatient population image to further warp the image to better fit theskull-stripped patient MR. At step 1308, the system may subsequentlyperform B-spline registration of the thus far warped image to furtherwarp the image to better fit the skull-stripped patient MR.

In an alternative example embodiment of the present invention, insteadof initially selecting just one patient population image to which toapply the one or more described registration and/or skull strippingprocedures, the system may initially select more than one, e.g., all ora subset including less than all, of the patient population images.According to an embodiment where a subset including less than all of thepatient population images is selected, the subset may be selected fromthe plurality based on the above-described image selection factors. Thesystem may apply the registration and/or skull stripping featuresdescribed above to all of the subset of the selected images.

The system may subsequently average the warped versions of the images toobtain the patient-specific atlas.

Alternatively, the system may determine a weighted average of the warpedversions of the selected patient population images. For determining theweights, the system may determine the similarity between the respectivepatient population images to which atlases correspond and the patientimage, and, based on the determination of the similarity, a degree towhich the patient population image should contribute to the final atlasregistration. For example, for this purpose, a mutual informationalgorithm may be performed to determine the similarity between theimages.

In an example embodiment, the image comparison algorithm may beperformed completely or primarily in the regions corresponding to thetarget regions. That is, even where the registered images are notsimilar overall, they may be similar in the relevant regions, and viceversa. Moreover, various target regions may be ranked, with higherweightings given to the higher ranked target regions. An overallsimilarity score may be calculated based on the similarities of thevarious target regions and/or the remaining regions, as modified by theweightings. The overall score may be used to determine the degree towhich the corresponding patient population image should contribute tothe final patient atlas. In an example embodiment, the overall imagesimilarity of the image as a whole may be additionally be factored intothe equation for generating the similarity score.

Feature Extraction

In an alternative example embodiment, the selected atlas, e.g., selectedbased on the factors described above, is registered to the MR image asfollows. First, the system finds certain predefined surfaces in the MRimage, e.g., those of the ventricles and/or Thalamus. For example, suchsurfaces may be found by the system by performing a pattern matchingalgorithm to find matches to corresponding predefined structures of theatlas. The system then automatically warps the corresponding predefinedsurfaces in the atlas to the identified surfaces of the MR image, e.g.,such that the warped three-dimensional atlas structures at leastapproximate the identified surface regions in the MR. The system maythen warp remaining portions of the atlas incidental to the warping ofthe regions corresponding to the predefined surfaces.

For example, the system may use a 3-D registration algorithm to minimizedifferences between 3-D data of the patient scan, and 3-D data of theatlas. For example, one such registration may include a non-rigidinter-subject brain surface registration using conformal structure andspherical thin-plate splines. However, other suitably appropriateregistrations may be performed instead. Remaining portions may then beincidentally modified automatically, e.g., according to the incidentalwarping method described in the '330, '312, '340, '343, and '314applications concerning FIGS. 9 and 10 of the '330, '312, '340, '343,and '314 applications. Thus, referring to FIG. 14, according to anexample embodiment of the present invention, the system may, at step1400, find predefined three-dimensional surface regions in a patientscan. At step 1402, the system may warp corresponding surface regions ofan atlas to match the identified surface regions of the patient scan. Atstep 1404, the system may warp portions of remaining regions of theatlas incidental to the warping of the predefined surface regions.

Alternatively, the automatic registration of the surfaces may determinean overall scaling/translation/rotation matrix to apply to the atlas asa whole. In an example embodiment, for determining the matrix to applybased on the automatic registration of the surfaces, different ones ofthe registered surfaces may be weighted differently in the calculationof the matrix. For example, certain ones of the registered surfaces maybe considered to be of greater importance than others in thedetermination of the atlas registration and may be set to have appliedthereto a greater weighting in the determination of the matrix. In anexample embodiment, the weightings may vary depending on the patient,e.g., depending on patient condition and/or the regions targeted forstimulation for the patient. (Different regions may similarly beweighted differently for determining a matrix for modifying one image tobe registered to another image.)

In an alternative example embodiment, the system may identify thepredefined surfaces and visually demarcate the surfaces in the patientimage, e.g., the MR image. A user may use a user interface to line upthe atlas with the MR image in a position where one of the atlas and MRimage overlies the other of the atlas and MR image. The system mayprovide atlas modification controls via which a user may shift thesurface regions of the atlas to at least approximately correspond to thepositions of the surfaces identified in the MR image, for example,according to the methods described in the '330, '312, '340, '343, and'314 applications concerning FIGS. 9 and 10 of the '330, '312, '340,'343, and '314 applications. Remaining portions may be incidentallymodified, e.g., according to the methods described in the '330, '312,'340, '343, and '314 applications concerning FIGS. 9 and 10 of the '330,'312, '340, '343, and '314 applications. Alternatively, the registrationof the surfaces as defined by the user's input, may determine an overallor local scaling/translation/rotation matrix, which the system mayautomatically apply to the atlas as a whole.

In an example embodiment of the present invention, the system mayautomatically determine how the atlas should be registered to the image(or how two images should be registered to each other) according tomethods described herein, or according to any suitably appropriatemethod. When the user performs the manual registration, the system mayoutput an indication of how well the items have been registered bycomparison to the automatically determined registration. The user canignore the indication or may further modify the registration based onthe indication. In response to such further modification, the system mayupdate the output indication of the degree of accuracy of theregistration. In an example embodiment, multiple such indications may beoutput, each for a respective anatomically significant region. Thoseregions considered to be significant for such output may depend on thepatient, e.g., based on the patient condition and/or those regions ofthe patient targeted for stimulation. In an example embodiment, the usermay input the regions for which such output should be provided. In anexample embodiment, different regions may be displayed in differentviews, and in one or more, e.g., each, of the views, the indicationsrelevant to the displayed regions may be output.

Auto-Determine a Lead Type and Hemisphere Using a Post-Op Image

The system analyzes a CT, MR, or an image of another imaging modality,to determine where a leadwire is located, including whether there isonly one leadwire or two or more. If there is only one leadwire, thesystem determines and records whether the leadwire is in the lefthemisphere or the right hemisphere. The system may identify the numberand type of leadwires implanted by searching for matches to particularpatterns in the post-op image. This may be useful for determining thecorrect number of programming interfaces to provide to the user via auser interface, e.g., one programming interface per lead, and thecorrect stimulation field models to apply when performing programming.The leadwire type and location may also be useful from a userperspective, when the user does not know how many leadwires areimplanted or where the leadwires are located. In an example embodiment,this information may be output to the user as a confirmation step at theconclusion of image analysis.

According to an example embodiment, leadwire type and/or location may bedetermined by stepping through a series of CT image slices and trackingchanges in image features generated by the leadwires. The determinationcan be made, for example, by removing all of the CT data but for thesaturated data, which would leave only or substantially only an outlineof the skull and a representation of the leadwire. The outline of theskull can be ignored by the system, e.g., since it does not match anexpected approximate shape for the leadwires, and/or recorded by thesystem as the skull.

FIG. 15 shows a series of images 1500A to 1500D representing CT slicestaken in the axial view. Each slice may include a brain outline 1550 andone or more leadwires 1500. On an actual CT, brain structures generallycorrespond to dark areas inside the brain outline 1550, whereas theleadwires 1500 tend to show as white dots. This is because of thedifferent densities of leadwires compared to the surrounding braintissue, i.e., the leadwires are much more dense than the brain tissue.Additionally, because the skull is also more dense than the brain, itwill be appreciated that ignoring the skull outline may facilitatedistinguishing of the leadwires.

As the slices progress from 1500A to 1500D, the leadwires 1500 may varyin cross section, e.g., decreasing in diameter towards the tip. It isnoted that FIG. 15 is not to scale, but is intended to show generallythat the size of the representations of a leadwire can vary betweenslices. Eventually, the dots corresponding to the leadwires 1500 eachmay be reduced in size to form a point at a slice corresponding to alocation near the very tip of the leadwire, and then disappear entirelyfrom subsequent axial image slices. Thus, the location of tips may bedetermined as corresponding to the slice(s) in which each respectiveleadwire forms a point. Additionally, the system may determine aleadwire trajectory based on how the leadwire cross sections move fromslice-to-slice. For example, a directional vector may be calculated thatconnects the center of each dot to represent the trajectory.

Although the leadwires were described as being represented as dots inthe axial view, it will be understood that the sagittal and coronalviews may have more than a dot representative of the leadwire in thoseslices through which the leadwire passes. For example, FIG. 16 shows aseries of images 1600A to 1600D representing CT slices in the sagittalview, where the leadwire cross sections may be substantiallycylindrical. Depending on the trajectory and/or shape of the leadwire,different portions of the leadwire are shown in each sagittal slice.Thus, slice 1600C shows only a tip-most portion of a leadwire 1610,whereas slice 1600B shows the entire length of the leadwire 1610. Theleadwires can be located in relation to the skull, which may beseparately detected as noted above, and/or may be located relative toother anatomical regions of the brain as determined by registration ofthe CT with an MR to which an atlas is registered.

The system may also automatically determine the type of the leadwire.For example, different leadwires may have different lengths. The systemmay include a database of leadwire types in association with theirrespective lengths. The system may calculate the length of a leadwirebased on the number of axial slices in which the leadwire appears,and/or based on its length in a coronal and/or sagittal slice. Thesystem may look up the database to determine the type by finding theleadwire-type having the associated length closest to the detectedlength of the leadwire. Alternatively, the database may associateleadwire types based on varying diameters and find the type having theclosest matching diameter to a stored leadtype.

In an alternative example embodiment, the system may determine the typeof a leadwire based on a pattern matching algorithm. For example, thesystem may store in a database leadwire types in association withrespective distinct patterns. Each leadwire type may have a differentcross-sectional shape in one or more of the three views. For example, anaxial cross-section of a leadwire having a single non-directionalcontact is easily distinguished from an axial cross-section of aleadwire having three directional contacts. The system may match thepatterns of the leadwires detected in the CTs to the stored patterns andidentify the leadwire as that associated in the database with theclosest matching pattern. It is noted that the CT detected leadwirepatterns may be different than an outline of the actual leadwires. Thatis, the CT images may include image artifacts such that a directcomparison to the actual outline may not be possible. Expected CTdetected patterns may be generated, for example, by averaging togethercorresponding slices of CT images containing known leadwire types.Accordingly, the system may store in the database the expected CTdetected patterns rather than the outlines. Alternatively, the outlinesmay be stored and the pattern-matching algorithms may match the detectedpattern to the expected detected patterns for the stored outlines. Thatis, the effects of artifacting may be known, so that the system may takepotential artifacting into consideration when comparing the detectedpatterns.

In an alternative example embodiment, markers which geometrically differbetween leadwire types may be placed on the leadwires. The system maydetect the geometric marker and match it to one stored in a database inassociation with a respective leadwire type, in order to determine theleadwire type. The detection of the geometric marker and the matchingmay be performed using a three-dimensional volume. For example, thesystem may store in a database a plurality of three-dimensionalrepresentations of different types of leadwires. The system may alsogenerate a three-dimensional representation of the imaged leadwire,e.g., for at least a portion of the leadwire that includes the geometricmarker. The stored representations may then be compared to the generatedrepresentation to determine a degree of fit between the representations.

In an example embodiment, the stored representation is overlaid onto thegenerated representation and the degree of fit is determined bycalculating a degree of overlap between the representations. Forexample, the system may conclude that the imaged leadwire is of the sametype as one of the stored leadwire representations where a thresholdamount of the representation of each is overlapped by the other.

In an alternative example embodiment, a non-electrode marker ispositioned on different leadwires at different distances to an adjacentelectrode. The system may detect the distance of the marker from itsnearest electrode. The system may match the detected distance torecorded distances in a database, each associated with a respectiveleadwire type.

Automatically Locate Leadwire Based on Registered CT Image andTrajectory and Depth Information

In an example embodiment of the present invention, the system mayautomatically identify the location of a leadwire based on (1) a CTimage taken prior to implantation of the leadwire, which CT image isregistered to an MR image in which anatomical structures may berecognized, and (2) information, e.g., input by a clinician, indicating(a) a ring and arc angle of a headframe used for insertion of theleadwire, (b) the depth of the leadwire insertion, and/or (c) theleadwire that is used. Information such as (a), (b) and (c) is typicallyinput during pre-surgical planning and stored, for example ashandwritten notes or stored into a clinician-accessable database or amemory device such as a CD-ROM, a DVD, a flash memory, etc. Theinformation may also be updated during surgery based on changes to asurgical plan, e.g., a new leadwire type, entry point or trajectory.After surgery, the information may be input into the system using e-mail(within a body of the e-mail or as an attachment), wirelesstransmission, the Internet or, in the case of a portablecomputer-readable storage medium, physically transferred via a mediareader such as a DVD drive. The system may determine the length of theleadwire based on the information concerning which leadwire was used.Alternatively, the system may make the determination without theinformation regarding the leadwire type, if the insertion depthinformation is a measure of the depth at which the bottom tip of theleadwire penetrates the anatomy.

For example, in a surgical planning stage, a leadwire trajectory may beselected. The trajectory may be for a leadwire to be implanted in abrain, and may be relative to a headframe, e.g., a headframe 1700, asshown in FIG. 17. The headframe 1700 may include a ring 1710 extendingwithin a plane approximately parallel to an axial brain slice. Theheadframe may further include an arc 1720 attached to the ring 1710. Thearc 1720 may be rotated about the ring 1710 to change the ring angle,and the insertion guide may be shifted along the arc 1720 to change thearc angle. The combination of the ring and arc angle may define aplanned trajectory 1730. Such information may be used by the system asthe trajectory information for determining the location of the leadwire.

In an example embodiment, for locating the headframe relative to thepatient's head, a CT image, taken after screws have been inserted intothe patient's head (e.g., at specific reference points on the head) viawhich the headframe is attachable to the patient's head, may beregistered to the MR image. Based on the location of the screws, theposition of the headframe, and thus the leadwire whose trajectory isdefined by angles of the headframe, relative to the MR image and itsincluded brain structures is known. To illustrate, in FIG. 17 the systemmay calculate a length of a leadwire 1740 and a location of its tip1750, based on an insertion depth and the trajectory 1730 input by theclinician. The system may then calculate a set of coordinates for theleadwire 1740 relative to the registered CT image, e.g., CT image spacecoordinates corresponding to the leadwire tip and a position along theleadwire shaft.

In an example embodiment of the present invention, arc and ring angle,and target location for end-point of leadwire may be user input directlyinto an image registration system and/or module and/or directly into astimulation programming system and/or module. In an alternative exampleembodiment, as noted above, arc and ring angle, and target location forend-point of leadwire may be input into a surgical implantation module,used for planning and conducting surgery for implanting the leadwire,and may imported into the image registration and/or programming systemand/or module. Alternatively, based on such information obtained in thesurgical implantation module, the system may record coordinates of theleadwire. Such coordinate information may be imported by, for example,the stimulation programming module, and used by the model for generatinga model of the leadwire positioned relative to anatomical structures ofthe brain. In this manner, the leadwire coordinates need not beextrapolated based on information subsequently input into the systemafter surgery has occurred. Instead, the information is made availableto the system from an early point in time, e.g., during planning. The CTimage may be registered to an MR image by the surgical implantationmodule, or may be imported by the programming (and/or registration)module for registration of the CT image with an MR image, e.g., usingmethods described above. (An atlas may also be registered to the MRimage, e.g., using methods described above, such that the leadwire modelmay be positioned relative to atlas features.)

Directional Auto Lead-Location

In an example embodiment, the system may automatically determine arotational position of a leadwire by analysis of shapes formed in CTimages of the patient at an anatomical region at which the leadwire ispositioned. The use of geometric markers as described above inconnection with the auto-determination of leadwire type, may alsofacilitate determination of leadwire directionality. For example, ageometric marker may generate a distinct CT pattern so that the patternindicates a single direction, e.g., a point along a circumference of aleadwire cross section on the CT images. Example geometric markers thatmay be used in conjunction with the system of the present invention aredescribed in the '330 application, in connection with FIGS. 24A-B, 25A-Band 26. In FIGS. 24A-B, the leadwire includes a pair of windows that canbe shifted relative to each other (e.g., rotated or offset). In FIGS.25A-B and 26, triangular marker bands are used to provide directionalreferences for determining the orientation of leadwire electrodes. Otherexample markers include a strip extending longitudinally down one sideof the leadwire or a such a strip with circumferential band extendingaround the leadwire, for example at or near each end of the strip. Suchcircumferential bands may be provided for proper alignment of the markerwith the leadwire, for ease of manufacturing.

In an example embodiment, the rotational location may be automaticallydetermined by comparing multiple shapes formed in different CT imageslices. The compared shapes may be formed in axial CT slices. However,slices from any of the viewing axes may be used in a similar fashion.

When comparing multiple shapes, one of the shapes may be a geometricmarker, e.g., a non-electrode element, and another of the shapes may bean electrode element. For example, FIG. 18 shows an axial CT slice of aleadwire 1800 in which three electrodes 1810/1812/1814 are arrangedabout the perimeter of the leadwire, e.g., at equal distances from eachother. FIG. 19 shows a different axial CT slice of the same leadwire1800. The cross section of FIG. 19 may, for example, correspond to aportion further from the tip than the portion to which the cross sectionof FIG. 18 corresponds. A non-electrode element 1910 may be arranged ata different level of the leadwire as shown below. The non-electrodeelement may be made of the same material as the electrodes. For example,it may be an unconnected, and non-functional, electrode, also known as adummy electrode. FIG. 21 shows an example profile view of the leadwire1800 along a longitudinal axis. The CT slice of FIG. 18 may be takenalong line A-A and the CT slice of FIG. 19 may be taken along line B-B.

As shown in FIG. 21, the non-electrode element 1910 includes aprotrusion 1912 that rotationally coincides with the electrode element1812. The protrusion 1912 serves as a distinguishing feature on thenon-electrode element 1910, as opposed to a remaining portion of thenon-electrode element 1910, which is substantially flush with a body ofthe leadwire. The non-electrode element 1910 may be rotationallypositioned about a center longitudinal axis of the leadwire such that itcoincides with the rotational position of one of the 3 electrodes1810/1812/1814. The artifact in the axial CT image slice caused by the 3electrodes 1810/1812/1814 may have a triangle-like shape, with threenoticeable primary vertices, each corresponding approximately to acenter of a respective electrode. The artifact in the axial CT imageslice caused by the non-electrode element 1910 may have an oval-likeshape including two noticeable vertices (or a rectangular-like shapewith two short sides) that are opposite each other. One of the verticescorresponds to the protrusion 1912, while the other extends in anopposite direction. Due to the rotational orientation of thenon-electrode element 1910 with respect to the center axis and relativeto the rotational orientation of the electrodes 1810/1812/1814 withrespect to the center axis, when the axial slice includingrepresentations of the 3 electrodes 1810/1812/1814 and the axial sliceincluding the representation of the non-electrode element 1910 are linedup, one of the two primary vertices of the slice of the non-electrodeelement 1910 will correspond with one of the vertices of thetriangle-like shape (e.g., the vertex corresponding to the electrode1812), while the other of the two primary vertices will not correspondto any vertex of the triangle-like shape, as shown in FIG. 20.

The system may therefore identify the electrode with which thenon-electrode lines up by identifying the vertex overlap. Based on theidentification of the electrodes, the system may properly set and/orcontrol which electrodes to turn on and/or the respective amplitudes tobe applied to the respective electrodes for a stimulation. For example,the system may determine which electrodes face which respectiveanatomical regions, and may accordingly operate the correct electrodesin the correct manner for producing the intended VOA. Similarly, thesystem may correctly rotationally model the leadwire relative to thepatient anatomy, so that a clinician may correctly input stimulationparameters associated with particular ones of the electrodes accordingto the positions of the electrodes relative to various anatomicalregions.

In an alternative example embodiment, instead of comparing the twoartifacts to see where primary vertices of each line up, the system maycombine the two artifacts into a single artifact by overlapping the twoartifacts. The system may then pattern match the combined artifact to astored pattern. Rotational positions of the electrodes may be identifiedrelative to the stored pattern. By aligning the combined CT artifact tothe stored pattern, the system may apply the rotational positionidentifications of the pattern to the aligned combined CT artifact,thereby identifying the rotational locations of each of the electrodes.

In an alternative example embodiment, instead of a non-electrode elementhaving a structure similar to that of the electrodes, a strip may extendlongitudinally down one side of the leadwire, which causes a CT artifacthaving a single point. The electrodes on either side of the strip, andthus also any remaining electrodes may be identified.

In an example embodiment of the present invention, the system mayprovide a user interface via which to obtain user-placed markers and/oruser-input location information the system may then use for determiningthe position and orientation of the leadwire. A user may manually selecta location of the tip of the leadwire, a point on its shaft, and a pointat which a directional marker is located. For example, these landmarksmay be recognizable by a user in displayed images. The user may place amarker at, or otherwise select, a region of an axial image slicecorresponding to a center of the lead tip, and the user may place amarker at, or otherwise select, a region of an axial image slicecorresponding to a center of one part of the shaft. The user may place amarker at, or otherwise select, a region of a coronal or sagittal imageslice corresponding to a region at which the marker on the leadwire islocated, thereby indicating the orientation of the leadwire. The systemmay then display a model of the leadwire according to the user-placedmarkers, within a representation of an anatomical volume of the patient.

Import Patient Data from an External Source

Patient data such as brain images (pre-implantation and/orpost-implantation), clinical notes, leadwire identification information,anatomical structure identification information (e.g., AC, PC, MCP andMSP locations) and stimulation programming settings may be stored on amemory device from an external source, which may be read by an imageregistration module and/or a stimulation programming module. Theregistration module may display the images in a user interface via whichdifferent images may be registered to each other. The stimulationprogramming module may display the images and estimated VOAs overlaidthereon. An example system according to the present invention is shownin FIG. 22, which includes a surgical planning system 2210, aprogramming and registration system 2220 and an implantable pulsegenerator (IPG) 2230. Each component 2210/2220/2230 may be inbi-directional communication with any other of the components. Thesurgical planning system 2210 may include a surgical planning module2212. The programming and registration system 2220 may include astimulation programming module 2222 and an image registration module2224. Each of the modules 2212/2222/2224 may be stored in a respectivedevice memory together with other modules or data.

In the example system of FIG. 22, the surgical planning system 2210 mayconstitute the external source. For example, the patient images may besaved onto a CD-ROM. A backup copy of the patient images may be storedin a database within the surgical planning system 2210. After thepatient is discharged, the patient may be provided with the CD-ROM forsubsequent use at a separate medical facility, e.g., a facilitycorresponding to the programming and registration system 2220.Alternatively, a clinician may directly transmit the patient images tothe programming and registration system 2220 via the Internet, e-mail, aprivate computer network, etc.

In an alternative embodiment, each of the modules may be co-located in asingle device. For purposes of illustration, the surgical planningsystem 2210 has been shown in FIG. 22 as external to the programming andregistration system 2220. However, the external device may be anyexternal source of patient images and may, for example, include otherprogramming devices and/or other surgical planning devices that are notpart of the same system.

The memory device may be an SD card, CD, etc. The images may betransferred via a network as well, e.g., using WiFi, bluetooth,hardwired, etc.

Integration Between Surgical Planning and Programming Systems

In embodiments in which the surgical planning system and the programmingsystem are separate systems, e.g., as shown in FIG. 22, informationinput during surgical planning, including information inputinter-operatively, may later be exported to the programming system. Forexample, a file may be created during the surgical planning stageincluding information regarding the coordinates of the anatomicalstructures of the patient's brain and the coordinates of the leadwire.For example, a brain atlas may be registered to a patient's MR.Additionally, the leadwire within the brain may be determined based on aselected arc angle and ring angle of a headframe attached to thepatient's head. A method for registering the atlas to the MR may includeselecting in the MR the AC, PC, and MSP, by which information the systemcan determine the way in which the atlas relates to the MR.

Alternatively, the CT and MR may be fused. The atlas may be registeredto the MR by selecting in the MR the AC, PC, and mid-sagittal line (orother anatomical landmarks). Then the leadwire can be located relativeto the atlas by selecting within the CT a termination point of theleadwire and a point on the shaft of the lead. Certain leadwires includea substantially rigid portion (e.g., a portion including the electrodecontacts) extending upward from the implanted tip of the leadwire, and amore flexible portion distal from the implanted tip. Since the flexibleportion may be bent, selection of a point along the flexible portion mayresult in an inaccurate trajectory determination. It may therefore bepreferable to select a shaft point on the rigid portion of the leadwire.

Either way, the above information may be stored using a surgicalplanning and performance module in a file that may betransmitted/provided to a separate programming system/module, e.g., bye-mail or physically removing a storage device from a first system andinserting it into the other system. Indeed, the features described abovewith respect to image registration/scaling, atlas registration/scaling,overlay controls, stem and pivot controls, scaling, feature extraction,etc. may be provided in an image registration/programming system or maybe provided in a surgical planning system, e.g., prior to a surgery inwhich a leadwire is implanted. Data obtained by use of such features ina surgical planning system may then be transferred to another system.Other information that may be transferred from a surgical planningsystem includes, for example, target and/or side effect VOAs, MER data(e.g., used for updating an atlas, which atlas information may also betransferred).

The programming system/module is configured for reading the filegenerated by the surgical module, and displaying graphical informationoverlaid on the registered MR and CT scans, with the registered model asobtained from the surgical module.

Export Patient Data to an External Source

Patient data, such as brain images (pre-implantation and/orpost-implantation), clinical notes, leadwire identification information,anatomical structure identification information (e.g., AC, PC, MCP andMSP locations) and stimulation programming settings, may be exportedfrom the registration/programmer module of one computer to that ofanother computer so that different clinicians can use the informationfor programming a patient.

In an example embodiment, the system is configured such that theinformation can be transferred to and obtained from the IPG so that theinformation travels with the patient. Whenever a computer having aregistration and/or programmer module links up with the IPG, it can viewthe information stored thereon for use to program the IPG for astimulation therapy.

Example information that can be transferred include program settings,registration information, including position of AC, position of PC,position of the MSP, leadwire tip, another point along the leadwireshaft, explored VOA regions, notes regarding the VOAs, etc.

The memory device used for the export may be an SD card, CD, etc. Thenetwork used for the export may be, e.g., using wifi, bluetooth,hardwired, etc.

The computer may allow the information to be modified and may store thenew information to the IPG, either as a new data file or by overwritingthe older information. In this manner, a clinician or other user of thecomputer may specify a new set of stimulation parameters for thepatient. The new stimulation parameters may be generated in response toa determination that the old stimulation parameters are no longereffective or need to be improved, e.g., due to changes in theorientation of the leadwire or the patient's anatomy or due to changesin patient condition. Other changes to the information include updatesto the explored VOA regions, along with notes regarding the exploredregions.

Select Target or Side Effect Volume Based on Image Registration to MRfrom a Patient Population

In an example embodiment, the system may store in a database MRs of apatient population, for whom efficacious volumes have been previouslydetermined. For example, stimulation parameters may be applied formembers of the patient population, for which application of stimulationparameters, the system may compute a respective estimated VOA. Further,information regarding the efficacy of the parameters, either sensed orreceived as user-input, may be obtained. The system may thus storeinformation indicative of which VOAs are efficacious for a plurality ofmembers of a patient population, further associated with respective MRs.

The system may select one of the members of the patient population onwhich to base a determination of a target VOA for the subject patient.Such selection may be made based on any similarity that is of clinicalsignificance, including similarities between the MRs as a whole,similarities between certain predefined portions of the MRs,similarities of MER data, of clinical profiles, ages, sex, etc. Withrespect to similarities between certain portions of the MRs, theportions on which to perform the image matching may depend on theclinical profile of the patient, as described above in regards toautomated atlas registration using rigid, affine, B-Spline registration.

The system may register the MR image of the selected member of thepatient population to the MR image of the subject patient.Alternatively, the system may select a subset of the members of thepatient population and register a composite image (e.g., an averageimage) formed by the MR images of the members of the subset to the MRimage of the subject patient. The selection of the subset may be basedon factors described in detail above in other discussions of a selectionof a subset of a patient population.

Registration of the MR images may be manual. For example, a clinicianmay interact with a user interface to overlay the two MR imagescorrectly aligned translationally and rotationally, and then scale,e.g., the selected MR image to the patient MR image, e.g., according tothe methods described in the '330, '312, '340, '343, and '314applications concerning FIGS. 9 and 10 of the '330, '312, '340, '343,and '314 applications, and/or by any other method described above forregistering an atlas to an MR image.

Alternatively, registration of the MR images may be performedautomatically using any of the methods described above for registeringan atlas to an MR image.

Once the MR images are registered, e.g., the user inputs informationindicating that the manual registration is completed, or navigates awayfrom a registration interface to a programming interface, or theautomatic registration is completed, the system may use the efficaciousVOAs of the member(s) of the patient population whose MR image has beenregistered as a target VOA in the registered space. For example, therecorded VOA may be warped along with its associated MR image to thesubject MR image, resulting in a target VOA within the space of thepatient MR image. Alternatively, a spatial relationship of the recordedVOA to the registered patient population MR image may be translated tothe warped version of the patient population MR image, thereby definingthe spatial relationship of the target VOA relative to the subjectpatient MR image.

In an alternative example embodiment, the MR images are not registeredto each other. Instead, once a member of the patient population isselected based on similarities to the subject patient, the system maytranslate the relationship of a VOA recorded for the member of thepatient population to predetermined structures within the MR image ofthe selected member to the structures within the MR image of the subjectpatient to form a target VOA for the subject patient. For example, thesystem may perform a surface extraction, as described above, in each ofthe MR images, and based on the position of the VOA relative to theextracted surfaces in the member MR image, the system may determine aVOA for the subject patient having the same relative position withrespect to the extracted surfaces in the patient MR image.Alternatively, the system may register an atlas to the member MR imageand to the patient MR image, and select as the target VOA, a VOA thathas a relative position to surrounding structures of the patient atlasthat is the same as the relative position of the VOA of the member ofthe patient population to the atlas structures of the atlas of themember of the patient population.

The determination of the target VOA for the subject patient need not bea one-time occurrence. Instead, new target VOAs may be generated for thesubject patient when, for example, a VOA not previously considered isfound to be particularly effective for at least one patient of thesubset of patients to which the subject patient has been registered, orfor at least one patient of a subset of patients who were associatedbased on similarities with the subject patient.

While the discussion above concerns determining a target VOA based onpreviously determined efficacious volumes of a patient population, thesystem may alternatively or additionally determine side effect volumes,e.g., which are to be avoided as much as possible during stimulation ofthe patient, based on recorded side effect volumes of the patientpopulation, by performing the steps described with respect to thedetermination of a target VOA.

Auto-Determine which Lead and IPG to Use Based on the Target LeadLocation, Trajectory, and VOA

Based on input regarding the target location at which the tip of theleadwire is to be located and the desired trajectory of the leadwire tothe target location, and further in view of a target volume ofactivation (VOA), the system may, e.g., during a surgical planningstage, determine and output a suggested leadwire to use and a suggestedimplantable pulse generator (IPG) to use as the source for thestimulation pulses (aside from outputting which stimulation parametersto use). Further, in an example embodiment, a user may input the desiredtrajectory without indicating the target at which the leadwire is toterminate in the patient, e.g., the patient's brain, and the system mayfurther output a suggested depth of implantation of the leadwire to beable to achieve an estimated VOA closest to a target VOA. The system maymake these determinations based on a stored atlas of anatomicalstructures, e.g., a brain atlas, registered to the patient's anatomy,relative to which the input trajectory is defined. For example, aheadframe may be positioned or located relative to the patient's head,and the trajectory may be defined by an arc and ring angle relative tothe headframe. The atlas may be registered, for example, to an MR imageof the patient, as described in detail with respect to the atlasregistration sections.

Certain leadwires have rotational electrodes that extend around theentire perimeter of the leadwire (i.e., non-directional electrodes),while others include a plurality of electrodes at a single cross-sectionof the leadwire, each extending about a respective portion of theperimeter of the leadwire that is less than the entire perimeter.

As for IPGs, certain IPGS are configured to turn on electrodes all atthe same amplitude, while other IPGs are configured to output differentamplitudes for different electrodes.

If the target VOA is positioned at one side of the leadwire, thenapplication of current at equal amounts all around the leadwire wouldproduce a VOA of which a large portion is to a side of the target VOA.For example, in FIG. 23, a leadwire 2300 having electrodes 2301 to 2306is positioned to one side of a target VOA 2310. The system may thereforerecommend a leadwire that has the multiple electrodes at a singlecross-section, and an IPG that is configured for applying differentamplitudes at different electrodes so that a current field may beproduced for producing a VOA predominantly extending from the leadwirebiased in certain directions, rather than equally about the leadwire, inorder to better match a target VOA that is biased towards certain sidesof the leadwire. For example, signals may be applied to electrodes2302/2303/2306 may be of higher amplitude than signals applied toelectrodes 2301/2304/2305. Further, among the electrodes 2302/2303/2306,the electrode 2303 may have the highest amplitude signal because it isnearest the center of the target VOA 2310.

The target VOA may be expressly input or may be determinedautomatically, e.g., based on input regarding the patient conditions andinformation regarding effective VOAs in a patient population.

The system may also provide a recommendation of which leadwire to usewhen there exist a plurality of potential leadwires. Referring to FIG.24, a set of leadwires 2401/2402 may be located within or near a targetVOA 2400. The system may also estimate VOAs based on various stimulationparameters to determine which of the leadwires 2401/2402, in combinationwith a particular IPG, provides an estimated VOA that best-matches thetarget VOA 2400. For example, the leadwire 2401 may provide a bettermatch because it has electrodes nearer a center of the VOA, and istherefore able to provide an estimated VOA covering more of the targetVOA 2400 for the same amount of signal (e.g., pulse, frequency, oramplitude) compared to an estimated VOA provided by the leadwire 2402.

The type of IPG used may also be taken into consideration. If an IPG isavailable that is capable of outputting signals at different levels fordifferent electrodes, the system may estimate VOAs for each leadwirebased on the application of the different signals to correspondingdirectional electrodes in the leadwires. However, if the IPG can onlyoutput a single level signal, the system may only estimate VOAs based onthe same signal being applied to each of the electrodes in a givenleadwire.

The results of estimating VOAs on the leadwires 2401/2402 may thereforeindicate that, although both leadwires 2401/2402 are capable ofproviding a matching estimated VOA given the right input(s), theleadwire 2401 is the superior choice because it would only require asingle-level output IPG with a lower power output, whereas the leadwire2402 would require a multi-level output IPG with a higher power output.

Display Representation of Volume Capable of Stimulation (VCS)

It may be unsafe to apply greater than a certain threshold amplitude.The system may graphically identify areas which are estimated to bestimulatable only with amplitudes that exceed a safe amplitude and/orthat are estimated as not being stimulatable with even the highestamplitude settings. For example, the system may grey out such a region,leaving in non-greyed out color the remaining region which is estimatedto be stimulatable with a safe amplitude setting. Display of such aregion may be useful for a clinician to determine regions on which tofocus for finding suitable VOAs and corresponding stimulationparameters.

Referring to FIG. 25, the system may display an estimated VOA 2501 basedon the application of specific stimulation parameters to a leadwire2500. A region 2502 extends inward from infinity towards the estimatedVOA 2501 and is bounded by a region 2503, which defines the maximumsafely stimulatable region.

In an example embodiment, the system may also determine whether theestimated VOA 2501 is fully-contained within the region 2503. Thisdetermination may be repeated as the clinician varies the stimulationparameters to simulate new VOAs. If an estimated VOA breaches the region2503, the system may highlight the overlapping portions, e.g., in red.

Pre-Compute VOA in a Background Thread

To avoid a delay between receipt of input of stimulation parameters andoutput of the estimated VOA corresponding to the input stimulationparameters, the system may pre-compute respective VOAs for certainstimulation parameter sets even before receipt of user input specifyingthose parameter sets. The pre-computed VOAs are stored in associationwith such parameter sets, and, in response to receipt of user inputspecifying such parameter sets, the system retrieves from memory anddisplays the pre-computed VOAs.

It is impractical to pre-compute VOAs for all possible parameter sets.Instead, the pre-computation is for a set of likely parameter sets to beuser-input, the likelihood being determined from the active parametersets or the last user-entered parameter sets.

Referring to, for example, FIG. 3g of the '330, '312, '340, '343, and'314 applications, the pre-computation may be for those parameters thatare one step removed in + and − directions for each of certain or alldirectional controls and/or controls for turning on and off theelectrodes. Each step may correspond to a predetermined amount of changein input, e.g., a 0.1 change in amplitude.

For example, for each active electrode of the last entered parameterset, there may be a respective arrow control to increase or decrease theamplitude of the applied current. Therefore, the processor mayprecompute VOAs for each parameter set where the amplitude for any oneof the electrodes is increased and where the amplitude for any one ofthe electrodes is decreased. For example, the top-right electrode isshown to be at amplitude of 2.7. The system may therefore pre-compute aVOA for where the amplitude of that electrode is 2.6 and for where theamplitude of that electrode is 2.8.

Additionally, arrow controls may be provided for shifting the currentfield as a whole, as formed by the combination of the various activatedelectrodes of the last entered stimulation settings, upwards, downwards,to the right, or to the left. Therefore, the system may pre-compute theVOAs for the settings corresponding to one such shift downwards from thecurrent settings, one such shift upwards from the current settings, onesuch shift to the right from the current settings, etc.

Additionally, certain electrodes may be on and others off. In an exampleembodiment, the system therefore also pre-computes the VOA for theamplitude setting that would be first set in response to the userselecting the electrode to be turned on. In an example embodiment, whenthe subject electrode is to be turned on, the system is configured toselect an amplitude setting for the subject electrode in view of theamplitude setting(s) of other electrodes in the same leadwire.

Compared to the present settings, the system may pre-compute the VOA forsettings corresponding to the turning on of any one of the electrodesthat are off according to the present settings. For leadwires withnon-directional electrodes, it may be assumed that the subject electrodewould be turned onto the same amplitude setting of an adjacent turned-onelectrode or, if no adjacent electrode is currently turned on, of theclosest turned-on electrode.

Additionally, for non-directional electrodes, the system may pre-computethe VOA assuming that the electrode would be turned onto the sameamplitude as that of a most recently modified neighboring electrode(modification can include the turning on of the electrode in addition tochanging its amplitude). For example, referring to FIG. 23, if theelectrode 2302 is to be turned on, and both the electrodes 2301/2303have already been turned on, the system may apply to the electrode 2302the same amplitude as that of whichever one of the electrodes 2301/2303was most recently modified.

For leadwires with directional electrodes located on the same level(e.g., electrodes at the same longitudinal distance along the leadwire),the system may set the amplitude of the subject electrode to be the sameas that of an activated electrode on the same level. Where there are aplurality of activated electrodes on the same level, or where there areno activated electrodes on the same level, the system may apply theprocedure described above, in which the amplitude of the most recentlymodified electrode is used. Referring to FIG. 23, if the electrode 2305is activated, the system may set the amplitude of the electrode 2302 tobe the same as the electrode 2305, even if one of the electrodes2301/2303 was turned on more recently than the electrode 2305. Thus, thesystem may accord priority to electrodes on the same level.

In an example embodiment of the present invention, when the leadwireincludes directional electrodes, the system may also be configured topre-compute the VOA assuming a clockwise or counterclockwise rotationalshifting of the entire field. Rotational shifting is described, forexample, in the '330, '312, '340, '343, and '314 applications inconnection with FIGS. 19 to 22. The system may pre-compute the VOA basedon a single clockwise and/or counterclockwise step (e.g., rotationallyshifting the inputs by one electrode).

The number of VOAs pre-computed in response to a change in user input,as well as the total number of stored pre-computed VOAs, may varydepending on hardware and timing constraints. For example, the number ofpre-computed VOAs may be a function of processor speed and/or memorysize. In one example embodiment, the system may maintain a cache forstoring the pre-computed VOAs, update the cache to include newlypre-computed VOAs, and to delete older pre-computed VOAs, e.g., on afirst-in-first-out basis.

Programming and Registration System

In an example embodiment, a system according to the present inventionmay include a patient registration system 2610 as shown in FIG. 26. Theregistration system 2610 may be communicatively connected to an IPG2680, which is in turn communicatively connected to a stimulationelectrode 2690. The registration system 2610 may implement any of themodules described above, and may include a processor 2612, a memorydevice 2614, a communication device 2618 and a user interface 2620.

The processor 2612 may be configured to execute instructions inaccordance with the various methods described above. The communicationdevice 2618 may be a media card reader, a telemetry device or any otherdevice by which the registration system 2610 communicates with externaldevices such as the IPG 2680. The user interface 2620 may include aninput device such as a keyboard or mouse, and an output device such as adisplay monitor.

The memory 2614 may include patient population data 2630, as well ascurrent patient data 2640. As described above, the patient populationdata may be used for atlas selection. Although shown separately, thecurrent patient data 2640 may be stored as a subset of the patientpopulation data 2630. The patient population data 2630 may includeseparate databases for various types of patient data, including a scanimage database 2631, an atlas database 2632 and a clinical profiledatabase 2633. While the patient population data 2630 is shown in FIG.26 as being a part of the system 2610, it may instead be storedexternally, at a central location accessible via a network by a numberof systems. Similarly, the current patient data may be exported to sucha central location for updating the patient population data 2630.

The current patient data 2640 may also include similar databases,including a scan image database 2641, a landmark database 2642 and aclinical profile database 2643. The scan image database 2641 includesfiles corresponding to CT, MR or other imaging modalities, taken beforeand/or after leadwire implantation. The landmark database 2642 mayinclude information designating the locations of various brainlandmarks, such as the AC, PC, MCP and MSP, relative to an imagecontained in the scan image database 2641. The clinical profile database2643 may include, for example, information about the current patient'smedical history, the IPG 2680 (e.g., a model number or a serial numberassigned to the IPG 2680) and/or the configuration of the stimulationelectrode 2690 (e.g., the number and type of the electrode contacts).

The memory 2614 may include any of the various modules described above,as well as additional modules for implementing any of the methods orsystem features described above. As shown in FIG. 26, this may includefor example a pivot/stem tool module 2620, a zoom tool module 2621, anMSP selection module 2622, an AC/PC selection module 2623, across-section ID module 2624, a slice scrolling module 2625, an atlasregistration module 2626 and an auto image correction module 2627.

System Integration

In an example embodiment of the present invention, systems may beprovided in a stand-alone version, where the settings in the stimulationprogramming module cannot be transferred from the module to the IPG, butinstead, the user would have to manually enter the settings the userlikes into another module that controls the IPG.

In an example embodiment of the present invention, systems may beprovided in a semi-integrated version, where a telemetry device, e.g.,the communication device 2618 of FIG. 26, is used for transportingstimulation parameters to the IPG for implementation thereof from astimulation programming module that computes and outputs an estimatedVOA for parameter settings, where the corresponding VOAs for the variousparameter settings are displayed for user review. When the user likes aparameter set, the user can input an instruction for the transmission ofthe settings to the IPG for implementation thereof.

In an example embodiment of the present invention, systems may beprovided in a more fully integrated version, where a telemetry device isused for transporting stimulation parameters to the IPG forimplementation thereof from a stimulation programming module thatcomputes and outputs an estimated VOA for parameter settings, where theparameters are sent automatically. For example, if the current patientor a patient in the patient population self-reports a particularside-effect or a benefit, or exhibits a measured side-effect/benefit inresponse to clinician-supervised testing, and/or in response to sensoroutput, the system may automatically determine a new set of stimulationparameter settings by adjusting the existing stimulation parameters,e.g., in step-wise fashion according to the VOA pre-computationdescribed above. The system may adjust the stimulation parametersettings so as to decrease activation in areas of the brain associatedwith the reported side effect or to increase activation in areas of thebrain associated with the reported benefit. As a safety measure, thesystem may only be allowed to automatically adjust the existingstimulation parameter settings within a predetermined range of parametervalues, e.g., a maximum allowable change in amplitude. The system mayalso be time-constrained by limiting automatic adjusting to a maximumallowable number of adjustments in a given time period, e.g., once perday, or to a mandatory waiting period between adjustments, e.g., twelvehours.

The above description is intended to be illustrative, and notrestrictive. Those skilled in the art can appreciate from the foregoingdescription that the present invention may be implemented in a varietyof forms, and that the various embodiments may be implemented alone orin combination. Therefore, while the embodiments of the presentinvention have been described in connection with particular examplesthereof, the true scope of the embodiments and/or methods of the presentinvention should not be so limited since other modifications will becomeapparent to the skilled practitioner upon a study of the drawings,specification, and following claims.

1. A system for providing image alignment data, the system comprising: acomputer processor configured to: display a first image, a firstposition of the image relative to an object being recorded; display animage positioning control including: a first edge positioned at a firstlocation of the image, a second edge, and a shaft extending between thefirst and second edges, the second edge being user-shiftable; andresponsive to user input to shift the second edge from a second locationto a third location: display the image in a second position relative tothe object, the second position being offset from the first position;and update the recorded position of the image to reflect the secondrelative position in a memory where the recorded position is accessiblefor generating, based on the recorded position, a model of an anatomicalregion.
 2. The system of claim 1, wherein at least one of the first andsecond edges is shiftable via a drag and drop action.
 3. The system ofclaim 1, wherein: the second and third locations are angularly offsetfrom each other about the first location; and the second position isrotationally offset from the first position about the first location. 4.The system of claim 3, wherein, responsive to user input to select andshift the first edge, the image is shifted relative to the object, suchthat the shifted first edge is at the first location of the shiftedimage.
 5. The system of claim 3, wherein, responsive to a user input todrag the second edge, the shaft is one of lengthened and shortened, theshaft length determining an extent of a linear positional offset of thesecond edge that is required for a corresponding rotation of the imageby a particular angular offset.
 6. The system of claim 3, wherein theimage is an image of an anatomical region obtained via a first imagingmodality and the object is an image of the anatomical region obtainedvia a second imaging modality.
 7. The system of claim 6, wherein thefirst imaging modality is magnetic resonance (MR) and the second imagingmodality is computed tomography (CT). 8-9. (canceled)
 10. The system ofclaim 9, wherein the anatomical region is of a brain and the object is arepresentation of a mid-sagittal plane. 11-25. (canceled)
 26. The systemof claim 1, wherein: the first image (a) is displayed in a firsttwo-dimensional view of a three-dimensional object, and (b) is of afirst two-dimensional plane of the three-dimensional object; theprocessor is configured to display, in a second two-dimensional view ofthe three-dimensional object a second image of a second two-dimensionalplane of the three-dimensional object; the second and third locationsare angularly offset from each other about the first location; thesecond position is rotationally offset from the first position about thefirst location; and responsive to the user input to shift the secondedge from the second location to the third location, the processor isfurther configured to at least one of: modify the second two-dimensionalview; and (i) remove the second image from the second two-dimensionalview, and (ii) display in the second two-dimensional view a third imagethat is of a third two-dimensional plane of the three-dimensionalobject, the second and third two-dimensional planes being orthogonal tothe first two-dimensional plane along a predetermined axis of the firsttwo-dimensional view.
 27. The system of claim 26, wherein the secondtwo-dimensional view is orthogonal to the first two-dimensional view.28-38. (canceled)
 39. A computer-implemented method for providing a userinterface for at least one of showing and setting relative positions ofobjects, the method comprising: displaying a flashlight object in adisplay area; displaying a first portion of a first object in a firstregion of the display area that extends in a predefined manner from theflashlight object; and displaying a first portion of a second object inone or more second regions of the display area that do not extend in thepredefined manner from the flashlight object.
 40. The method of claim39, wherein the flashlight object is user-shiftable to shift the firstregion, thereby responsively: bringing into view a second portion of thefirst object; removing from view at least a part of the first portion ofthe first object; bringing into view a second portion of the secondobject; and removing from view at least a part of the first portion ofthe second object.
 41. The method of claim 40, wherein at least one of:(a) positions of the first and second objects relative to each other,and (b) the portion of one of the first and second object beingdisplayed in the display area are modifiable in response to a same userinteraction within the display area, the modifiability of the relativepositions being at least one of rotational and translational; and thefirst and second objects are different images of a same anatomicalregion.
 42. (canceled)
 43. The method of claim 40, wherein: a size ofthe flashlight object is user-modifiable; and at least one of: the sizemodification correspondingly modifies a size of the first region, anamount of the first object that is displayed in the display area, and anamount of the second object that is displayed in the display area; andgraphical protrusions extending from opposite edges of the flashlightobject are user-shiftable towards a center of the flashlight object andaway from the center of the flashlight object, for, respectively,decreasing the size of the flashlight object and increasing the size ofthe flashlight object. 44-71. (canceled)
 72. A non-transitivecomputer-readable medium having stored thereon instructions executableby a processor, the instructions which, when executed by the processor,cause the processor to perform a method for associating a region with apredefined landmark, the method comprising: displaying a firsttwo-dimensional plane of a three-dimensional object; displaying a secondtwo-dimensional plane of the three-dimensional object; displaying afirst user-selectable and shiftable marker associated by the processorwith a landmark; and responsive to a user-shift of the first marker to aregion of the first two-dimensional plane: displaying a second markerassociated with the landmark in a region of the second two-dimensionalplane that corresponds to the region of the first two-dimensional plane;and recording an association of the landmark with each of the regions.73. The computer-readable medium of claim 72, wherein: the secondtwo-dimensional plane is substantially orthogonal to the firsttwo-dimensional plane; and the region of the second two-dimensionalplane provides a substantially orthogonal view of the region of thefirst two-dimensional plane.
 74. The computer-readable medium of claim73, wherein: the orthogonality of the second two-dimensional plane tothe first two-dimensional plane is along a line extending betweenpredefined points of a first display area in which the first twodimensional plane is displayed; the first two-dimensional plane isuser-rotatable within the first display area and relative to the line,thereby modifying where along the first two-dimensional plane the lineextends; and the method further comprises, responsive to a rotation ofthe first two-dimensional plane within the first display area, modifyinga second display area in which the second two-dimensional plane wasdisplayed to display a different two-dimensional plane orthogonal to therotated first two-dimensional plane along the line.
 75. Thecomputer-readable medium of claim 73, wherein at least one of: thethree-dimensional object is one of: anatomical, the first and secondtwo-dimensional planes being two-dimensional images of the anatomicalobject; and a region targeted for a stimulation and of which the firstand second two-dimensional planes are two-dimensional representations;and the landmark is an anterior commissure, and the method furthercomprises displaying a third marker, the third marker beinguser-selectable and shiftable and associated by the processor with aposterior commissure. 76-79. (canceled)
 80. The computer-readable mediumof claim 73, wherein the method further comprises, responsive to asubsequent user-shift of the first marker to a second region of thefirst two-dimensional plane that does not have a corresponding region inthe second two-dimensional plane occurs, removing the second marker fromdisplay.
 81. The computer-readable medium of claim 73, wherein at leastone of: in a first display area in which the first two-dimensional planeis displayed, two-dimensional planes of the three-dimensional object arescrollable between the first two-dimensional plane and othertwo-dimensional planes that are parallel to the first two-dimensionalplane, in response to which scrolling the processor adjusts thebrightness of the first marker so that it gradually fades as a distancebetween the scrolled-to two-dimensional plane in the first display areaand the first two-dimensional plane increases; and (a) the methodfurther comprises, responsive to a user-input zoom instruction,magnifying at least a portion of the first two-dimensional plane that iswithin the first marker, and (b) at least one of: the at least theportion of the first two-dimensional plane that is magnified isdisplayed within a demarcated magnification window, the magnificationwindow having protruding therefrom a first user-selectable control, inresponse to selection of which magnification is increased, and themagnification window having protruding therefrom a seconduser-selectable control, in response to selection of which magnificationis decreased; and responsive to a scrolling instruction for scrollingfrom the first two-dimensional plane to another two-dimensional planethat is parallel to the first two-dimensional plane, the scrollinginstruction received while the at least the portion of the firsttwo-dimensional plane is magnified, the processor, executing theinstructions, at least one of (a) modifies the recorded association ofthe landmark to be with a region in the scrolled-to two-dimensionalplane and not with the region of the first two-dimensional plane, and(b) automatically shifts the second marker to another region of thesecond two-dimensional plane. 82-91. (canceled)